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Executive Summary 


State sector education policy in England aims to deliver raised standards of attainment and 
equality of educational opportunity by offering fair access to schools for all pupils from any 
background. During the lifetime of the previous Labour government (May 1997 to April 2010) a 
key policy tool used to tackle entrenched low levels of academic performance among state 
secondary schools in areas of decline has been the school renewal initiative of the Academies 
Programme. This scheme was first announced in March 2000 and came into operation in public 
sector secondary schools from September 2002, with the opening of 3 renewed schools in that 
year. By end-April 2010, a total of 203 Academies had been established. 

In its original form, the Academies Programme has involved the rejuvenation of a failing 
secondary school through giving a private sponsor the flexibility to adopt innovative approaches 
in the running of the rebuilt and rebranded institution. Additionally, Academies have acquired an 
independent status which grants them autonomy from the local authority control characteristic of 
most state secondary schools. From the outset, the aims of the initiative have been: (1) to drive up 
overall standards of school performance, including raising the achievement and aspirations of 
underprivileged pupils in deprived areas; (2) to enhance local choice and diversity in the 
provision of state schooling through offering new techniques of education delivery; and (3) to 
feature a more inclusive and mixed-ability background of pupils within the Academy school. 

In this paper the effectiveness of the Academies model of school improvement through 
institutional transformation is assessed with specific reference to whether the scheme has been 
capable of delivering inclusive access for disadvantaged pupils in poor areas (aim (3)). Empirical 
evaluation looks at how the pupil profile of Academies changed once they opened under their 
renewed school type. Pupil-level data contained in the National Pupil Database and school-level 
data derived from various sources is used to consider (i) how the academic quality and 
composition of pupils entering year 7 of Academies and (ii) how the whole school composition of 
Academies have differed from both those in their predecessor versions and in similar schools that 
did not convert to Academy status. The methodological approach taken is that of a difference -in- 
differences analysis applied to a sample consisting of 33 Academy schools (five cohorts) and 326 
non- Academy schools over an 11 year period of available data, 1997 to 2007. 



Results indicate that the Academies Programme is failing some disadvantaged pupils, precisely 
the group the original scheme has aimed to cater for. Academy conversion is associated with a 
school performance-favouring change in the pupil profile of these institutions, an outcome that is 
in direct contrast to objective (3). Key findings are:- 

• There is an immediate jump up in the academic quality of pupils entering year 7 of the 
Academies sample once the policy comes into effect. Academies admit pupils into year 7 
with a Key Stage 2 end-of-primary- school attainment record that is 2.409 total points 
higher on average. This is a statistically significant and robustly identified finding that 
remains even after accounting for a potentially larger pupil capacity in Academies. 

• There is some evidence to suggest that the entry of higher ability pupils to Academies has 
been made possible by a reduction in intake numbers at the lower end of the attainment 
distribution. Evaluation shows that once schools converted into Academies they reduced 
their intake ability dispersion by 0.514 standard deviation units, implying that the 
attainment profile of pupils entering these schools reflects a more ‘exclusive’ intake. 

• Intake into Academies has consisted of a lower proportion of pupils from relatively 
deprived backgrounds, measured by those who are eligible for free school meals. In the 
Academy years the average percentage of free school meal eligible pupils in year 7 fell by 
5.563 percentage points to 38.61% (a drop of 12.59%). Again this finding is statistically 
significant and remains even after controlling for school size changes in Academies. 

Taken together, these results suggest that the Labour government’s programme of school 
conversion into an Academy has featured a relative rise in stratification within the schooling 
system compared to that which went before, implying a worsening of education inequality. Since 
the Coalition party came into power in May 2010 there has been a marked shift in the direction 
and core focus of the Academies Programme. All state primary, secondary and special schools 
throughout England can now convert to Academy status, while secondary schools with 
outstanding pupil performance in age- 16 tests have been able to gain priority fast-track 
conversion. The outcomes of the original policy indicate the consequences of this revised version. 
Aspects like independence from local authority control coupled with a continued pursuit of 
academic excellence may encourage newer Academies to adapt their admissions towards a more 
homogeneous and advantageous pupil intake, a fragmented situation that would further reduce 
fairness in access to schools, lowering potential attainment and educational opportunity among 
disadvantaged pupils in particular. 



Are England's Academies More Inclusive or More 
'Exclusive'? The Impact of Institutional Change on the Pupil 

Profile of Schools 


Joan Wilson 

1. Introduction 1 

2. Labour's Academies Programme: Structure, Aims and Effectiveness 7 

Brief history 7 

Key features 7 

Aims and Objectives 14 

Academy Schools and GCSE Performance 18 

3. Outlining the Empirical Process 21 

Academy schools sample 21 

Data description 24 

Dataset construction 26 

Empirical methodology 30 

4. Empirical Results: Main Findings 38 

Robustness checks 44 

Dynamic effects 53 

Possible mechanisms behind changing intake quality 56 

5. Empirical Results: Further Findings 63 

Assessing other dimensions of intake quality and examining changes 
in whole-school composition 63 

6. Summary and Discussion 66 

References 70 

Tables 74 

Figures 91 

Appendices 93 



Acknowledgments 


Dr. Joan Wilson is a researcher at the LSE Public Policy Group and was formerly a 
member of the Education and Skills team at the Centre for Economic Performance, LSE. 

The author would like to thank the following for their much valued advice and 
comments: Olmo Silva, her PhD supervisors Heather Joshi and Kirstine Hansen, and also 
thanks to Stephen Machin. Permission to use the data contained in this paper was kindly 
granted by the formerly named Department for Children, Schools and Families. This 
research forms part of the author's ESRC-funded PhD thesis, the doctorate for which was 
awarded in March 2010. 



1 


Introduction 


State sector education policy in England aims to deliver raised standards of attainment and 
equality of educational opportunity through offering fair access to schools for all pupils 
irrespective of their background and geographical location. The 1988 Education Reform Act lay 
the foundations for the formation of a quasi-market in the provision of state education, by giving 
pupils the possibility of choosing the school attended and allowing a scheme of competition 
between schools for pupils to be established as means for developing higher achievement and 
more equal access. The Act also implemented long-term changes in the way the education budget 
is transferred to schools in order to grant institutions more autonomy over their financial 
operations. Self-management was put in place through enabling the governing bodies of 
individual schools to receive their annual revenue funding direct from central government as 
opposed to through usual Local Education Authority (LEA) administration (Machin and 
Vignoles, 2005). At the same time a system of pupil-led funding began to evolve, in which the 
monies passed on to maintained schools have been made to more closely take account of the 
background circumstances and quantity of their pupil base, under a ‘fair funding formula’ 
determined by the LEA (West and Pennell, 1997; Sibieta et al., 2008). 

During the lifetime of the former Labour government (May 1997 to April 2010) per-pupil 
spending in the state schools sector rose by 6.4 per cent per year net of inflation, while overall 
expenditure on publicly-provided education saw real term increases averaging about 4.3 per cent 
per year from 2000 (Holmlund et al., 2009; Sibieta et al., 2008). Despite this rise in the real value 
of education funding - coupled with increasing school budgetary control and policies that sought 
to generate market-like competition - the education policy stance of the previous ruling party 
involved tackling the existence of a persistent tail of underperforming state secondary schools at 
the bottom end of the attainment distribution. In their 1997 election manifesto, Labour launched 
an attack on low state school standards, adopting a “zero tolerance of underperformance” 
approach to dealing with the issue (Labour Party, 1997). Historically, low-attaining secondary 
schools have featured heavily in deprived areas and are largely responsible for providing 
education to pupils characterised by social and economic disadvantage. Sustained failure in 
schools at the secondary education phase maintains the problem of education inequality and the 
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presence of a pool of ‘hard-to-reach’ pupils whose situation of deprivation and disadvantage 
continues into adulthood (Machin and Vignoles, 2005; Machin et al., 2007). 


As part of a strategy to improve school standards, policy initiatives revolving around institutional 
change as a means for school renewal have been instigated since 1997, following the introduction 
of Fresh Start schools 1 2 . One particular high-profile scheme that was announced by the Labour 
government in March 2000 and came into operation in state secondary schools from September 
2002 is that of the Academies programme. In its original form, this initiative involved the 
rejuvenation of a failing secondary school in an area of disadvantage through delegation of school 
control to a private sponsor. The Academy sponsor has the flexibility to adopt innovative 
approaches in the running of the school (initially this was in return for a committed financial 
contribution), in an attempt at reforming the school into a viably competitive education provider. 
Crucially, Academy schools are granted exemption from the LEA control that is characteristic of 
most state secondary schools and they instead have acquired an independent status. As a result 
they have a greater degree of input associated with determining facets of their functioning, 
including their staffing, subject teaching, and admissions rules, particularly those applied when 
there is above-capacity demand for entry to the school - subject to compliance with the legal 
requirements of the School Admissions Code. On the whole the scheme has sought to achieve 
three main aims from its outset: (1) to drive up overall standards of school performance, 
including raising the achievement and aspirations of underprivileged pupils in deprived areas; (2) 
to enhance local choice and diversity in the provision of state schooling through the use of new 
techniques of education delivery; and (3) to feature a more inclusive and mixed-ability 
background of pupils within the Academy school. By end- April 2010, a total of 203 Academies 
had been established (Machin and Vemoit, 2010a), with plans to extend their coverage to 15 per 
cent of secondary school education provision by 2015". The scheme had received a greater 


1 The Fresh Start initiative was first introduced by the Labour government in 1997 in the White Paper Excellence in 
Schools as part of the school improvement policy agenda. It was applied to underperforming schools - those 
classified as being in ‘special measures’ - from 1998 onwards. More specifically, “where schools over three 
consecutive years failed to get 5 good GCSE [General Certificate of Secondary Education] passes for at least 15 per 
cent of their pupils, they would be considered for a Fresh Start” (DfES, 2000). Very often it acted as a last resort for 
failing schools often characterised by a high fraction of pupils from low socio-economic backgrounds with lower- 
than-average academic outcomes. The policy involved the closure of a failing school, the employment of new school 
staff and the development of a renewed school ethos, plus either a complete refurbishment of the physical plant of 
the school or the continued use of existing school buildings. The initiative was partially modelled on the American 
schools model of ‘Reconstitution’, under which failing schools were started from afresh with new staff, new 
leadership and a new curriculum (Matthews and Kinchington, 2006). In total, only 37 Fresh Start schools were 
established in England by 2008 (Hansard, 2008d). From their inception Academy schools were hailed as a “new 
approach”, bringing “a radical new edge to the Fresh Start initiative - strengthening the programme designed to turn 
failure into school improvement” (DfES, 2000) (see also Wilson, 2009 for further details). 

2 The Academies scheme initially came with a government target of 200 Academy schools to be either fully open or 
in the pipeline by 2010, 60 of which were to be in London (DfES, 2004, pp. 9, paragraph 6). In November 2006 this 
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platform of significance following Labour’s announcement of the National Challenge in 2008, a 
new target system of achievement requiring all schools to have at least 30 per cent of their pupils 
attaining five or more A*-C General Certificates of Secondary Education, or GCSEs (including 
English and Maths) by 2011. Under this initiative all such weak schools were to be given the 
option to convert to an Academy school (DCSF, 2008; Curtis et al., 2008). 

In this paper the effectiveness of institutional transformation in the form of the original 
Academies model is assessed with specific reference to whether the scheme has been capable of 
delivering inclusive access to the renewed school for pupils in disadvantaged areas (aim (3)), in 

a 

light of the requirement of these schools to raise educational standards of attainment (aim (1)) . 
School reconstitution can be expected to positively affect the schooling and life chances of pupils 
if it results in the provision of better quality education and more schooling options for all at the 
expense of none. On the other hand school improvement initiatives that result in increased 
stratification along the lines of pupil ability and pupil characteristics will worsen education 
inequality. The Academies programme of the Labour government is of particular significance in 
this respect as the popularity of these schools increased during their time in power, with 
applications for places frequently exceeding school capacity, suggesting that as these new types 
of schools re-established themselves they may have been more able to ‘cherry pick’ pupils to 
enter the school from the pool of applicants. More specifically, Academies may have pursued a 
more favourable intake and composition of pupils beginning their secondary phase of education, 
with a school-performance-enhancing slant towards entry by those of higher prior ability and 
from a relatively better-off family background, to the detriment of less well-off and academically 
weaker pupils comprising their conventional admissions type. Equally, the renewed institutions 
may have aimed to acquire an improved pupil composition within the school as a whole in order 
to secure and maintain a stronger record of performance at the expense of wide-ranging inclusion. 
Such outcomes call into question the capability of a scheme that has sought to offer school 
improvement to a target group, through increased access to potentially better quality schooling, to 
bring about change to that audience. In turn these outcomes produce uncertainty over the extent 
to which the initiative has been able to achieve both its specific objective of greater inclusion and 


was doubled to 400 Academies that had been predicted as being established by 2015 (HC, 2008; Rogers and 
Migniuolo, 2007). Labour projected that the number of state secondary schools in the system would be 2,659 by the 
academic year 2014-2015 (HC, 2008). 

1 All details presented in this paper are from the ‘Key Concepts: School Choice and School Improvement’ section 
and Part II (chapters three to five) of the author’s PhD thesis. This was submitted in September 2009, the viva 
examination was passed in December 2009 and the final thesis was completed and submitted in March 2010 (see 

Wilson, 2009). 
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the more general education policy aims of raising standards of academic attainment and reducing 
education inequality. 

Delivery on goal (3) of Labour’s Academies programme is evaluated here by looking at how the 
pupil profile of Academy schools changed once they opened under their renewed school type. In 
this respect, pupil-level data contained in the National Pupil Database and school-level data 
derived from various sources is used to consider (i) how the academic quality and composition of 
pupils entering year 7 of Academy schools has differed from both that in their predecessor 
versions and in other similar schools that did not convert to Academy status; and (ii) how the 
whole school composition of Academies has differed from that in their pre-Academy versions 
and in comparison schools. The methodological approach to empirical evaluation is that of a 
difference-in-differences analysis applied to a sample of Academy and non-Academy schools 
over an 11 year period of available data, 1997 to 2007. Findings reveal an immediate jump up in 
the academic quality of pupils entering year 7 of the Academies sample once the policy comes 
into effect. There is some evidence to suggest that entry into Academies of higher ability pupils 
has occurred to the detriment of academically weaker pupils. Furthermore, intake into Academies 
has consisted of a lower proportion of pupils from relatively worse-off backgrounds. These 
results indicate unequal access to the renewed schools for pupils from deprived backgrounds with 
a lower historical education performance, thereby suggesting that the renewed schools have failed 
in their delivery of goal (3) and are more ‘exclusive’ post-policy. Thus the Labour government’s 
programme of school conversion into an Academy seems to have featured a relative rise in 
stratification within the schooling system compared to that which went before, implying a 
worsening of education inequality. 

A potentially more stratified education system resulting from the original Academies model also 
presents a plausible negative outcome arising from the version of this model recently developed 
by the Coalition party. The general elections of May 2010 saw the replacement of the ruling 
Labour party with a Coalition government (comprising of Conservatives and Liberal Democrats), 
whose education policy has been marked by a significant shift in the direction and focus of the 
Academy schools programme, details of which are contained in the Academies Act 2010 4 . As is 
the case for existing Academies, the Act allows institutions subscribing to the scheme to be 

4 Details of the Coalition’s Academies Act can be found on the Department for Education (DfE) website (accessed 
25 February 2011): http://www.education.gov.uk/schools/leadership/typesofschools/academies/a0061222/academies- 
act-2010 . Note that the Coalition government established the DfE in May 2010. This replaces the Department for 
Education and Skills (DfES) and the Department for Children, Schools and Families (DCSF), both of which were 
names given to the central government education department when the Labour party was in power, the former in the 
time of Tony Blair and the latter when Gordon Brown took over party leadership in June 2007. 
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autonomous from local authority control, enabling greater operational independence. However, 
the new policy has also witnessed a major extension that has taken its scope of coverage in all 
directions, with a particular initial emphasis on status change among schools at the upper end of 
the attainment distribution. Specifically, the Act has put in place the legislative groundwork for 
all state primary, secondary and special schools throughout England to convert to Academy 
status. Secondary schools deemed to be outstanding in terms of the performance of their pupils in 
age- 16 examinations (GCSEs) have been able to gain priority fast-track conversion status. At the 
same time, these outperforming institutions are required to enter into partnerships with weaker- 
attaining maintained secondary schools in their area in order to help boost their standards. Thus, 
while the policy caters to some degree for the tail-end of public sector secondary schools, it 
places the majority of immediate emphasis on Academy conversion among high- achieving state 
schools in this phase of education, so that overall the vastly revised initiative represents a 
momentous divergence from the original programme that had at its core a focus on improvement 
among failing secondary schools in particular and the underprivileged pupils in areas of decline 
who typically attend these schools 5 . Aspects such as independence from local authority control 
coupled with a continued pursuit of academic excellence may encourage newer Academies to 
adapt their admissions towards a more homogeneous and advantageous pupil intake, a 
fragmented situation that would produce less fairness in access to schools, lowering potential 
attainment and educational opportunity among disadvantaged pupils. Thus findings from the 
analysis presented here have implications for education policy that go well beyond those 
attributable to the original form of this school renewal initiative, mattering also for the latest take 
on this scheme by indicating one possible dimension of its likely consequences. 

The outline of the rest of this paper is as follows: Section 2 presents a short history of the 
Academies programme and outlines the key features of Academy schools. The objectives of the 
scheme since its inception are also set out, while the capacity of the initiative to have achieved 
one of its key aims of inclusion is discussed as a motivation behind empirical evaluation. 
Additionally, evidence on what is known about the effectiveness of the programme so far is 
presented here, focussing on standards of age 16 examination attainment in Academy schools. 

Section 3 sets the scene for the empirical focus on the effectiveness of school improvement. 
Details on the Academies that opened between the academic years 2002/03 and 2006/07 are 

5 The Academies of the Coalition government aim to improve standards for all pupils, to narrow the education 
inequality gap, and to provide world class schools - see http://www.bbc.co.uk/news/10161371 (accessed 25 February 
2011 ). 
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presented, where these schools form the five cohorts for assessment. Then the data to be analysed 
in this paper is introduced and explored, with the main data source being the National Pupil 
Database, indicators from which are combined with those in school-level data files. The dataset 
construction section includes the procedure behind the formation of a balanced panel of 
observations used to analyse intake composition changes in state secondary schools over 2001/02 
to 2006/07 and intake quality and whole school composition patterns over the 1 1 year window of 
1996/97 to 2006/07. Further to this comes a description of the empirical strategy employed - that 
of difference-in-differences regression estimation combined with propensity score evaluation. 
This approach enables estimation of an ‘Academy effect’ on intake and composition changes to 
be determined and compared to both changes in predecessor schools and also those in a set of 
control schools contained within an identified region of common support. 

Section 4 includes the main results from statistical regression analysis. Empirical findings on KS2 
intake quality changes in Academy schools relative to comparison groups are laid out. A host of 
robustness checks are presented, which explore whether the estimated post-conversion jump up in 
intake quality captures the actual policy effect. The notion of heterogeneous responses to the 
policy by the Academy cohort is then examined through a series of dynamic effect model 
specifications. Further to this, the possible mechanisms driving the change to intake quality are 
looked at, and evidence on changes in the dispersion of intake is put forward. Section 5 assesses 
other dimensions of compositional variation in Academies relative to predecessors and non- 
Academies, including changes in the proportion of pupils admitted to the renewed schools who 
are eligible for free school meals. Finally, Section 6 concludes with a summary and discussion of 
the work presented in this paper, in which some thoughts are offered on the effectiveness of this 
scheme of institutional change in enhancing equality in educational opportunity through fair 
access. Furthermore, the implications of the results from evaluation are considered in light of the 
significant changes made to the Academies Programme by the Coalition government. 
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2 Labour’s Academies Programme: Structure, Aims and Effectiveness 


Brief istory 

As is the case for many of the more recent education policy initiatives, the legislative origins of 
the Academies programme lie in the 1988 Education Reform Act. It was in this Act that a new 
type of state secondary school, the City Technology College (CTC) was introduced. These non- 
fee-charging institutions represented the very first type of specialist school of their kind as they 
were particularly oriented towards teaching the subject of technology. CTCs combined autonomy 
from LEA control with a path-breaking initial implementation of public-private collaboration in 
state education, involving as they did business or voluntary sector sponsorship (Astle and Ryan, 
2008). CTCs provided the initial legislative framework for the introduction of Academies, which 
were first launched onto the secondary schools arena by the then Labour government in March 
2000 in a speech on transforming the secondary phase of education, given by the former 
Secretary of State for Education, David Blunkett 6 7 . The first three Academy schools officially 
opened early on in the 2002/2003 academic year and over time the scheme witnessed steady 
growth followed by a flourish of heightened activity prior to the change in government in May 
2010. Academies, like CTCs, were originally described as “independent state schools” (Curtis et 
al., 2008, pp. 22, in reference to the then Prime Minister, Tony Blair). The key features of these 
distinctive institutions that led to this term are set out below, and are compared to those of 
traditional state secondary schools. 


Key eatures 


(i) Autonomy - In contrast to other state schools, Labour’s Academies were set up so as to be 

' 7 

fully managed by their governing body and they remain independent from LEA control . As a 
result the LEA has no direct funding link to these Academies as it does for all maintained schools 


6 The actual legislation for the formation of Academies is contained in the 2000 Learning and Skills Act (Astle and 
Ryan, 2008). 

7 As has been the case for CTCs, Academy school independence has not meant that parents of pupils attending these 
renewed schools have had to pay fees, since Academies are in the state schooling sector, where education provision 
is free. Instead fee-charging occurs within private schools in England, independently-run institutions that are 
autonomous from any form of state control. 
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in its jurisdiction. Instead Academy funding is channelled straight from central government as a 
block grant, an aspect that can reduce transaction costs in their financial management, with the 
resultant savings said to enable higher per pupil funding and teacher salaries (Gadkowski, 2007). 

(ii) Governance - Under the original scheme, conversion to Academy status led to the governing 
body of the school being created afresh. Small in size, there are anywhere between 6 to 16 
governors on the board of the established schools, though it is common to have 13 members, the 
majority of whom (usually around seven) were appointed by the Academy sponsor upon 
conversion, subject to approval by the central government education department (Astle and Ryan, 
2008) * 8 . Stakeholder governors have featured heavily on the governing body so that sponsor 
representatives have been able to “determine the ethos and leadership of the academy, and ensure 
clear responsibility and accountability.” 9 Early Academies were not required to appoint elected 
community or staff representatives to their governing body, nor were they required to have more 
than one elected parent governor and one LEA representative (Rogers and Migniuolo, 2007). The 
operations of the governing body are contained within each individual Academy’s Funding 
Agreement, legal documentation that was drawn up between the school and the education 
department of the Labour government (Astle and Ryan, 2008). In comparison, in LEA-controlled 
schools the governing body comprises of both appointed and elected representatives (Gadkowski, 
2007). Of these, Community schools tend to have higher LEA representation; Voluntary- 
controlled (VC), Voluntary- aided (VA) and Foundation schools with a Foundation contain 
representatives from the Foundation Body on their governing board (Goodwin, 2007). Hence the 
governance structure of Academies has given them management autonomy, with the majority 
sponsor-appointed board of governors largely holding responsibility for steering the operations of 
the school. 

(iii) Sponsorship - Non-govemmental sponsorship is a feature of Labour’s Academies. Sponsors 
have originated from a number of different fields such as business, religious organisations, the 
voluntary and charitable sectors and individual philanthropy. School sponsorship has arisen either 
through government invitation or otherwise interested parties have independently put themselves 
forward to get involved in the scheme (Gadkowski, 2007). In earlier versions of the programme, 
sponsors, in return for a financial contribution to the Academy, entered into a schools partnership 
with the government, and were granted management control of the school as well as the freedom 


See also http://www.standards.dfes.gov.uk/academies/what are academies/organisation/?version=l (accessed 21 

August 2008). 

9 See http://www.standards.dcsf.gov.uk/academies/what are academies/?version=l (accessed 21 August 2008). 
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to shape aspects of the school through the Funding Agreement 10 (see (zv) below). Specifically, 
sponsors have been able to influence factors such as the curriculum, through introducing 
innovative curriculum practices, and the choice of subject(s) in which the school specialises. 
They are also able to make their mark in areas like the pupil learning behaviour policy (which 
includes discipline), governance rules and admissions procedures in the event of place 
oversubscription (Gadkowski, 2007). In terms of the maintained schools sector, specialist schools 
are also sponsored, though their LEA control means that the influence of the sponsor is much 
weaker in comparison to that of Academy school contributors (Curtis et al., 2008). 

(iv) Financing and Buildings - Capital financing of Academies was the original means by which 
an Academy sponsor contributed to the school and justified their permitted input into school 
functioning. Sponsors put forward the lesser of £2 million or 10 per cent of capital costs towards 
the development of a new or refurbished Academy school building, payable over the lifetime of 
the building project. The substantial remaining construction expenses * 11 were initially covered by 
the Labour government through their school capital expenditure scheme that provided finance for 
the building of 1,100 new schools over a decade spanning 1997 to 2007 (Astle and Ryan, 2008). 
Subsequent Academy builds have been covered entirely by the Building Schools for the Future 
(BSF) capital programme, which has provided finance for school constructions undertaken from 
2005/06, although under the new government this has since changed . The sponsor’s capital 
contribution was replaced by an endowment fund to go towards expenses that are unrelated to the 
school build, but more recently Labour had planned to remove the financial contribution 
requirement altogether so as to facilitate broader sponsorship interest in the scheme . 


10 Funding agreements have tended to omit any detailed targets relating to the academic performance of the Academy 

(Gadkowski. 2007). 

11 The National Audit Office evaluated the cost of 26 out of 27 of the Academy schools that opened between school 
years 2002/03 and 2005/06 and estimated that Academies had cost around £24 million to build on average, and 
around £27 million for a completely new build. These figures compare with costs of £20-£22 million for other (non- 
academy) new secondary schools, representing as much as a near 17% lower cost. (NAO, 2007). 

12 On 5 July 2010 the newly-established Coalition government announced an overhaul in the building programme of 

England’s schools, including an ending of the BSF programme of capital financing (see 

http://www.education.gov.uk/schools/adminandfinance/schoolscapital/funding/bsf/a0061486/overhaul-to-englands- 
school-building-programme accessed 18 February 2011). 

13 In July 2006 an endowment model of sponsorship was introduced. Here sponsor proceeds of £2 million would go 
into a charitable endowment fund, the payment of which was normally expected to be spread over 5 years with an 
initial fee of £500,000 due in the first year. Disposal of this endowment was to be undertaken by the Academy trust 
and was to be spent on measures aimed at counteracting “the educational impact of disadvantage and deprivation 
and/or for educational work within the local community” (Rogers and Migniuolo, 2007, pp. 10). In 2009 the then 
Labour government announced that new Academy sponsors would no longer be required to make a financial 
contribution to the school, and this was to apply to Academies opening from September 2011 (see 
http://www.dcsf.gov.uk/pns/DisplavPN.cgi7pn id=2009 0158 accessed 8 September 2009). 
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The financing of all non-capital costs relating to Labour’s Academy schools has come entirely 
from central government. They have included an initial start-up grant for books, materials and 
classroom equipment, where this has been calculated according to the expected pupil capacity in 
the Academy and is mostly paid during the first year of opening. Academies opening in 2008/09 
received an average funding of about £874,000 through this grant. Additionally, Academies have 
been eligible for a grant to cover transitional costs and financial outlays associated with the 
process of Academy preparation. This fund is made available over the first two to three years of 
Academy school opening, or longer if the Academy did not replace any predecessor school(s). 
For Academies opening in 2008/09 grant awards of this kind averaged around £969,000, but with 
considerable variation across Academies, some receiving as little as £123,000 and others as much 
as £3.2 million (Flansard, 2008b). 

Coverage of the running costs of the school has come under a “general annual grant” which the 
Academy received directly from the Secretary of State under Labour. Funding allocated to 
existing Academies is calculated according to the LEA’s funding formula, yet it also includes an 
additional allowance that is equivalent to the money that the LEA does not usually pass on to 
maintained schools. This means that Academy school governors, as the administrators of the 
school’s finances, manage a higher proportion of their budget than do LEA-governed schools and 
it appears that they receive a greater budget overall, factors which give them greater financial 
freedoms. However the Labour government did state its commitment to reaching parity of 
funding between Academies and other maintained schools in the same area facing similar 
circumstances to Academies. The general annual grant has offered further provisions for a per 
pupil allowance for Academy schools with specialist status, though this is funding which all 
specialist schools, including maintained specialist schools, are entitled to 14 . 

(v) Admissions - Independence from the LEA in Academy schools means that the governing 
body is the admissions authority in these institutions. Details on the admissions policy are 
contained within the Funding Agreement of each of the established Academies. Where an 
Academy has replaced a predecessor school or schools, most pupils from the old school(s) have 
been expected to be given the option of readmission to the Academy school 15 . Since the 2002 

14 See http://www.standards.dcsf.gov.uk/academies/what are academies/funding/?version=l (accessed 21 August 
2008). The funding that the LEA withholds from maintained schools reflects expenses that go towards the payment 
of central services such as Pupil Referral Units (PRUs) and costs associated with SEN provision (Sibieta el al., 
2008). LEAs do not control established Academy schools and it is likely that any central services required by the 
Academy can be paid for directly. Therefore the Academy school share of these withheld funds can go straight into 
Academies, increasing their budget. 

15 See http://www.standards.dcsf. gov.uk/academies/faq/?version=l#582283 (accessed 21 August 2008). 
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Education Act, Academies have been able to acquire specialist school status in one or more 
subjects so that, like maintained specialist schools (of which almost 90 per cent of state secondary 
schools are), they can reserve up to 10 per cent of their intake for pupils with an aptitude or 
ability in the school’s specialism(s) (Gadkowski, 2007; Astle and Ryan, 2008; Smithers and 
Robinson, 2009 16 ). However, selection of this ki nd has only been permitted where the school has 
opted to specialise in particular subjects, namely sports or physical education (PE), the visual 
arts, the performing arts, modern foreign languages, information technology and design and 
technology . In practice most specialist schools, including currently active Academies, do not 
undertake admissions selection based on some indicator of talent in the specialism (Smithers and 
Robinson, 2009). Gadkowski (2007) reviewed the Funding Agreements of 46 Academies that 
opened between September 2002 and September 2006 and found that, of these, only 6 operated 
priority entry to the school according to specialism knowledge. Academy schools were originally 
described as being “fully inclusive all ability schools” that must comply with the School 
Admissions Code; under Labour enforcement of this was assigned a responsibility of the 
Secretary of State for Education 18 . 

In comparison, in Community and VC schools admissions decisions are in the hands of the LEA, 
while VA and Foundation schools are, like Academies, their own admissions authority. All LEA- 
maintained schools are also required to comply with the School Admissions Code, enforcement 
of which is carried out by the schools adjudicator. While existing Academy schools have only 
been required to be involved in local admissions forums, Community, VA, VC and Foundation 
schools must all participate in coordinated admission systems across the LEA (Goodwin, 2007). 
In the event of oversubscription to the school, Academies, like Foundation and VA schools, have 
been able to set their own oversubscription admissions conditions and this is done according to 
ranking categories determined by the Academy sponsor(s). Preferential entry based on measures 
of proficiency in the school’s specialism and place allocation through the grouping of pupils into 
admissions bands are two commonly identified procedures that can be used alone or 
conjunctionally (Gadkowski, 2007; Hansard, 2008a). 

The two distinct aspects of Labour’s Academy schools mentioned here - namely their 
independence from LEA control and their discretion to set their own admissions arrangements 

16 Between 1994 and 2008 a total of 2,688 out of 3,073 state secondary schools were designated as specialist, 
representing 87.5% overall (where the figure of 3,073 schools excludes those with a sixth form, CTCs and 
Academies) (Smithers and Robinson, 2009). 

17 See http://www. standards. dcsf.gov.uk/academies/faq/?version=l#582277 (accessed 21 August 2008). 

18 See http://www.standards.dcsf.gov.uk/academies/what are academies/management/?version=l (accessed 21 
August 2008). 
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within compliance of the legal requirements of the Code 19 - suggest room for variation in intake 
patterns following school conversion to Academy status. This is because predecessor schools 
were not organised along these more autonomous lines. These specific features, together with the 
stated aims of the original Programme (outlined in the ‘Aims and Objectives’ section below), 
provide the motivations for comparing the composition of pupils entering these Academy schools 
with that of pupils entering the predecessor version(s) in particular. 


(vi) Staffing - In established Academies appointment of the school principal has been in the 
hands of the sponsor(s) initially and after that the governing body, while school governors have 
taken full responsibility for the employment of school staff. In maintained schools there is more 
LEA involvement in both head teacher and staff appointment, though governor input in these 
matters takes place in VA and Foundation schools (Goodwin, 2007). Labour’s Academy schools 
have a far greater degree of flexibility over staff employment contracts relative to LEA-controlled 
schools. The governing body of the Academy is able to authorise any changes to the terms and 
conditions of employment relating to hired personnel and has ultimate responsibility for the 
approval of personnel practices concerning matters such as staff development and discipline . 
Academies have not been required to follow national frameworks relating to staff pay and 
conditions" . However, despite these freedoms, most staff from the predecessor school(s) have 
been expected to transfer to the new Academy school under the 1981 Transfer of Undertakings 
(Protection of Employment) or TUPE regulations, in which case their existing terms and 
conditions of employment are upheld. Otherwise, a common variant of staff terms used by 
existing Academies has involved lengthening the working day, or year, or both (Rogers and 
Migniuolo, 2007) Additionally, these Academies have been able to operate performance related 
pay measures including the payment of bonuses to their staff for good performance; they may 


19 For the sample of Academy schools (as well as their predecessors and non-Academies) analysed here, the 
applicable School Admissions Codes are those of 1999 and 2003, which cover the admissions period September 
2000 to August 2007, after which point the 2007 and 2009 Codes came into effect for September 2007 and 
September 2009 admissions respectively. The 1999 and 2003 Codes came attached with fewer statutory adherence 
requirements than subsequent versions: they contained admissions guidelines, which admissions authorities for 
schools (the LEA or a school’s governing body) were only required to “have regard to” (DfES, 2003, pp. 40, 
paragraph A.l). Thus for the period September 2000 to August 2007 schools who were their own admissions 
authority had more discretion to decide on who to admit to the school, both under normal conditions and in the event 
of oversubscription, so long as procedures adopted were not unlawful (West et al., 2009; Wilson, 2009, Chapter One, 
Section 1.4.1). 

20 See http://www.standards.dcsf.gov.uk/academies/what are academies/organisation/?version=l#1576175 
(accessed 21 August 2008). 

21 Specifically, Labour’s Academies have not had to follow the School Teachers’ Pay and Conditions Document 
(STPCD) or the national framework of service conditions for school teachers in England and Wales, known as the 
Burgundy book. The STPCD is a legally enforced document that establishes teacher pay scales, rules for promotion 
and working time, professional tasks, and absence cover conditions, among other issues. The Burgundy book 
additionally sets out illness pay, maternity pay and notice to leave requirements. In practice pay scales in Academies 
tend to closely replicate those in the STPCD (Sibieta et al., 2008). 
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also offer other financial incentives such as childcare subsidies and contributions to relocation 
expenses (Astle and Ryan, 2008). 

(vii) Accountability - The governing bodies of existing Academy schools have been directly 
accountable to the Secretary of State for Education in the main, though they are additionally 
answerable to local parents. The Secretary of State must approve any policy change requests by 
the Academy that relate to its admissions, SEN, learning behaviour or terms of governance, as 
contained in each Academy’s Funding Agreement. The governing body of a maintained school is 
considered to be more accountable to local parents. As for all state schools, Academies are 
inspected by the Office for Standards in Education (OfSted) school inspections body whose job it 
is to monitor and ensure their compliance with national standards of education provision. Once 
opened, Academies were fully inspected within one to three years, more commonly in their third 
year (Gadkowski, 2007), although less formal monitoring visits also took place soon after the 
initial open date of the Academy (Curtis et al., 2008). 

(viii) Curriculum - Earlier cohorts of Academies (those existing prior to the Summer of 2007 ) 
were not required to adhere to complete teachings of the National Curriculum as are other state 
schools. Instead their curriculum was to be broad and balanced, only requiring teaching and 
assessment in the core subjects of English, maths and science at Key Stage 3 (when pupils are 
aged 13/14). Curriculum innovation has been encouraged in Labour’s Academies programme, 
and the governors and senior managers of established schools have the flexibility to develop a 
curriculum catering for the needs of individual low-attaining pupils. Additionally, Academy 
schools are specialist schools and as such their curriculum includes a focus on the chosen 
specialist subject(s) (Gadkowski, 2007). 

Overall, there are many and varied differences between Academies and other schools in the state 
sector, which revolve in the main around the concept of independence, and give rise to the 
classification of Academies as “independent state schools”. In the next section the aims of the 
distinctive Academies model developed by the previous Labour government are set out in detail 
and the means by which the features of these schools have been expected to help them deliver on 
their goals are discussed. 


22 See http://www.standards.dcsf.gov.uk/academies/what are academies/cucciculum/?version=l (accessed 21 
August 2008). 
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Aims and bjectives 


In February 2003 the education department of the former Labour government commissioned 
PricewaterhouseCoopers (PwC) to undertake an independent five-year evaluation of the 
Academies programme and to produce a report for each year, the first of which was available in 
November 2003 (Rogers and Migniuolo, 2007). This report sets out the three “ultimate” 
objectives of the scheme from its inception:- 

(1) “Academies will contribute to driving up standards by raising achievement levels for their 
own pupils, their family of schools and the wider community by breaking the cycle of 
underachievement and low aspirations in areas of deprivation with historical low 
performance; 

(2) Academies will be part of local strategies to increase choice and diversity in education. 
They will have innovative approaches to one or more of governance, curriculum, staffing 
structures and pay, teaching and leaming[,] structure of the school day and year[,] using 
ICT [Information and Communications Technology]; and 

(3) Academies will be inclusive, mixed ability school[s]” 23 

Originally, the Academies programme concerned the replacement of “seriously failing schools” , 
in which the underachieving predecessor school or schools that went before were to be rebuilt 
and rebranded into an Academy. In this respect Academies were to establish themselves “where 
significant changes in the nature and management of schools were needed” (DfES, 2000). 
Otherwise the initial Academy set-up was to involve a new school development in an area of 
sustained low educational attainment. Hence objective (1) emphasises that Academy schools 
aimed to play a key role in community regeneration. From the outset these schools were posited 
as a means for tackling educational underperformance and “establishing a culture of ambition to 
replace the poverty of aspiration that was generally there before” (Adonis, 2008, pp. 15). Their 


23 PwC (2003, pp. Al). See also Curtis et al. (2008). 

24 Curtis et al., 2008, pp. 14, quoting a speech by the then Education Secretary David Blunkett in March 2000. 
Failing schools were initially defined as those “which are either in special measures or underachieving” (DfES, 
2000). One Academy school could replace more than one pre-existing failing school at a time, though the rebuild 
usually used the existing land site of either of the failing schools being replaced. Originally, the Academies model 
was applied to cities; hence the term “City Academies” was used in reference to these new types of school. The 
prefix ‘city’ was dropped in the 2002 Education Act, when the policy was extended to include non-urban areas 
(Curtis et al., 2008; Education Act, 2002). 
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formation was to target areas characterised by a historic trend of disadvantage and decline, with 
Academies considered as having the capacity to transform the education experiences of children 
in these areas. More specifically, Labour’s Academy schools have been largely intended to cater 
for pupils most exposed to the local area deprivation that represents a feature of their 
establishment, so that it is likely that the social background and educational attainment of pupils 
attending these schools reflects the relatively deprived circumstances of the area they inhabit. 

Objective (2) places Academies in the realm of choice-oriented education policies. They were 
viewed as being schools of innovation, designed to generate institutional competition, resulting in 
a diversification in the supply of state-funded education at the local level. This implies an 
inadequacy in available provisions at that time, a gap that was to be filled by a new type of school 
run along more autonomous lines than those afforded to traditional state schools. 

Elaboration on the meaning behind objective (3) is given in the 2002 Education Act, where it is 
stated that an inclusive, diverse-ability Academy school is one that “provides education for pupils 
of different abilities who are wholly or mainly drawn from the area in which the school is 
situated” (Education Act, 2002, Section 65, 2(b)). Like the first objective then, aim (3) 
emphasises that Academy schools were expected to be at the forefront of local improvement. 
Academies were to incorporate a varied spectrum of pupil types, with pupil admissions taken 
predominantly from the local supply pool, so that the characteristics of their composition should 
largely reflect the demographics of the local pupil population. 

The nature of the original Academies model was such that it was perceived as being able to attain 
the first objective. The new school building resulting from the scheme was anticipated to offer a 
flagship feature of symbolic value that could contribute to raised expectations of change and 
provide a visible demonstration of local community investment and reform taking place (Curtis 
et. al . , 2008; Astle and Ryan, 2008). This redevelopment of school facilities has aimed to foster a 
pupil’s motivation to learn, encouraging both their own and their parents’ commitment to and 
involvement in maintaining standards of quality and performance in the school. Sponsorship of 
the school by private business, voluntary or religious sector members was also considered a 
means by which standards could be raised. The assumption was that a sponsor could bring a 
vision and values to the school that could both define and renew its ethos. His or her business 
experience, expertise and network of contacts could serve to strengthen the integration of the 
school into the local community, and position the academy sponsor as an adult role model for 
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pupils in economically and socially deprived areas. In terms of the functioning of the school, 
academy autonomy from LEA control provides a means for allowing sponsors the “freedoms and 
flexibilities” (Rogers and Migniuolo, 2007, pp. 27) to challenge traditional lines of schooling 
operations and introduce innovative practices into the school in a bid to raise performance. The 
sponsor is able to shape the way things are done in the academy through his or her personal and 
potentially unique contribution to the Funding Agreement, in which the organisation of the school 
in relation to aspects such as the curriculum, governance, admissions and discipline are outlined 
(Gadkowski, 2007). In general, it is the various institutional arrangements of Labour’s Academies 
model - such as their curriculum innovation, accountability, staffing and funding autonomy, their 
new school building, and the unique input of an Academy sponsor into the school - that policy- 
makers expected would provide the mechanisms through which performance improvements 
would be triggered. 

In terms of objective (2), the independence of an Academy, its use of innovative techniques and 
the collaboration with non-government organisations that the original programme involved were 
all designed to serve the purpose of creating a new approach to education provision and an 
alternative type of state-funded education in the secondary schooling arena. The notion that an 
Academy school could inject further choice and a diversity of supply into state education thus 
relates to the ability of this schooling model to rejuvenate a failing, unpopular pre-existing school 
with spare capacity and reintroduce it to the quasi-market place as a viable, in demand, education 
provider. An increased diversification in the local mix of schools brought on by the successful 
Academy status restart of a predecessor school was presumed to encourage “more competition 
and contestability which can lift performance in an area” (PSA Delivery Agreements, 2008, pp. 
9), suggesting another means by which the Academies model was considered able to achieve the 
local area benefits of the first aim. 

In respect of goal (3), a potentially higher pupil capacity in an Academy was to provide one 
channel through which the school could be expected to incorporate a more inclusive and socially 
diverse range of pupils. Places offered at the Academy were to be greater in number to the extent 
that the new school building or the remodelled version could accommodate a larger quantity of 
pupils than the predecessor school(s). Another means for achieving this goal has come through 
the admissions rule of ‘banding’, which existing Academies can apply only when they are 
oversubscribed (DfES, 2003). This method of ranking place allocation “is generally taken to 
mean selecting an intake so that its spread of ability is representative of a wider population. This 
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wider population could be all the applicants to a particular school or group of schools, or the 
whole pupil population in a geographical area such as a local authority or nationally” (Tough and 
Brooks, 2007, pp. 19). The process “involves testing all children applying for a school place and 
placing them into ability bands as a result of the test” (DfES, 2003, pp. 16, paragraph 3.27). This 
is therefore an additional aspect of Academy school functioning that was anticipated to enable 
them to cater to the final objective. 

A system of expanded school capacity and oversubscription rules that were intended to offer fair 
chances of admission to pupils from across the ability range might have ensured a more balanced 
academic intake into Labour’s Academies and may have allowed these schools to be more 
inclusive without changing the quality distribution of their pupil entrants. On the other hand, the 
requirement of Academies to raise achievement standards could have created an incentive for 
these schools to try to adopt more ‘exclusive’ admissions practices and skew their intake 
distribution towards students with a historically high level of attainment and associated social 
characteristics, so as to make the task of driving up performance in the Academy school easier. In 
the time over which the Labour party were in power it was well documented that Academies were 
becoming increasingly popular, with their admissions demand exceeding available places at the 
schools. The central government education department noted that “Academies overall are three 
times oversubscribed. The brand new Academies, without an underperforming predecessor 
school, have nearly six applicants for every place. Academies directly replacing previously 
underperforming schools have more than two applicants for each place, and are now filling nearly 
25% more places than the schools they replaced.” Oversubscription was bringing with it interest 
in Academy admission by a different class of pupils, a new direction that was openly welcomed 
by the Minister for Schools who was responsible for the programme at that time: “The popularity 
of academies extends across all classes and I welcome this. I want academies to be socially mixed 
schools attractive to the middle class” (Adonis, 2008, pp. 8). Thus Academy schools were 
broadening their appeal to include a particular genre of pupils that had not been a prominent 
feature of predecessor school(s), while at the same time Academies were starting to face capacity 
pressures as a direct consequence of their heightened status. These situations, together with the 
fact that established Academies are their own admissions authority, offer preliminary suggestions 


23 See http://www.standards.dcsf.gov.uk/academies/what are academies/working/?version=l (accessed 21 August 
2008) 
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of changes in intake patterns in the renewed schools relative to those in the previous LEA- 
governed school(s), and in particular they imply a shift towards a favourably ‘exclusive’ intake 26 . 

The aim of this study is to compare the pupil profile of Labour’s Academy schools to that in both 
predecessor institutions and similar schools that did not turn into Academies over the time period 
of available data. Evaluation considers the prior attainment and background composition of year 
7 entrants - and aspects of whole school-level composition - in Academies relative to these other 
schools over the years 1997 to 2007 . Thus the purpose of evaluation is to determine the extent to 
which aim (3) of the original Academies Programme in particular has been satisfied. To date no 
previous research taking an identical empirical approach exists in this specific area . The 
competition effects of Academies, implicated by aim ( 2 ), have also not yet been assessed and this 
type of analysis provides a potential area for future research. In terms of objective (1), 
preliminary analysis that considers the academic performance of early cohorts of Labour’s 
Academies at the GCSE stage relative to achievement levels of their predecessors has been 
conducted. Findings from this research are summarised below in order to provide some initial 
information on what is known about the effectiveness of the programme as it originally stood. 


Academy chools and GCSE erformance 

Recent co-authored research (Machin and Wilson, 2008) conducted a school-level analysis of 
changes in GCSE performance in Labour’s Academy schools, in order to evaluate the 
effectiveness of the scheme in delivering its explicit aim of raising standards in education. This 
goal, as was mentioned above, was presumed to be deliverable through the private sponsorship 
aspect of the Academies Programme in particular and the freedoms granted to the Academy 
sponsor to introduce innovative techniques into the running of the school, including a business- 
like system of school management and governance. 


J ' In Appendix 7, the prior school types of schools that converted to Academies are discussed, in reference to the 
sample of 33 Academies that form the basis of this research. About 72% and 3% of Academy predecessors were 
Community and Voluntary-controlled schools respectively in this sample. These schools are characterised by 
majority-LEA representation on the school governing body, such that the LEA was the admissions authority for most 
of the Academy predecessors. 

27 In their recently released work, Machin and Vernoit (2011) use the unique approach of comparing outcome 
variables in Academy schools to a particular set of control schools, namely those state sector schools that represent 
post-sample period future Academies. They find similar - though weaker - results on changes to intake quality as 
those outlined in this paper under their analytical method. 
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The study considered Academy schools opening in their new status between September 2002 and 
September 2005, thus including four cohorts of 27 Academies in total. The methodological 
approach taken was that of statistical difference-in-differences estimation, in which the pre-policy 
school-level GCSE attainment of Academy predecessors was contrasted with the GCSE 
performance of these schools in the effective years of the policy, and this difference was set 
against that in two groups of comparison schools. The first group consisted of matched schools, 
one per Academy, where the matching school was identified as one within the LEA of an 
Academy, sharing similar pre-policy levels and trends in GSCE performance as the Academy, but 
without itself acquiring Academy status. The second group included all other state secondary 
schools in the Academy school’s LEA. The purpose behind establishing a unique group of 
matched schools in particular was to enable assessment of the impact of a school becoming an 
Academy on GCSE achievement with unobservable school-level components that might explain 
some of the measured result netted out. 

Estimation utilised 11 years of school-level records of GCSE attainment, covering the period 
1995/96 to 2005/06, where attainment was measured by the percentage of pupils getting 5 or 
more A*-C grades at the GCSE stage (when pupils are aged 15/16). Table 1 shows the results 
from difference-in-differences regression analysis that compares changes in GCSE outcomes over 
the pre -policy and post-policy years in Academies relative to that in both matched schools (Panel 
A) and other LEA secondary schools (Panel B) 

Across almost all cohorts there is no evidence of a positive ‘Academy effect’ on GCSE 
performance for schools that switched to Academy status. This is not the case for cohort 3 
Academies (opening from September 2004), when they are compared to the matched set of 
schools (columns 5 and 6). The percentage of pupils achieving 5 or more GCSEs graded A*-C is 
10 percentage points higher in the effective policy years for this cohort (and is 8.95 percentage 
points higher when school-level time-varying controls are added to the regression), suggesting 
that GCSE attainment improved relative to the predecessor years of the schools. However, given 
that there are only five Academy schools in this cohort, this finding provides at best a weak 
indication of performance improvements in the renewed schools as a whole. 

More recent research has indicated potential GCSE attainment gains among Academy schools 
that have been open for a longer time period. Machin and Vernoit (2010b) determine the impact 
of conversion into an Academy on school performance by comparing the pre-and-post conversion 
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GCSE attainment of existing Academies to that in a selected counterfactual group of schools due 
to open as Academies in the future. This difference-in-differences comparison enables evaluation 
to consider what the attainment outcome would have been for current Academies if they had not 
become Academies. Their preliminary evidence has revealed that, for Academies that have been 
open for two years or more, “an extra 3% of pupils in the academies are achieving at least five or 
more grades A*-C at GCSE/GNVQ compared with the schools that have not yet become 
academies” (ibid., pp. 20). The authors have found no evidence of performance improvements 
among Academies that have been open for less than two years, offering one reason as to why the 
earlier study by Machin and Wilson (2008) drew no positive attainment effects of Academy 
status. They comment that their results, although preliminary and inconclusive, offer some signs 
“that academy schools can deliver faster gains in GCSE performance than comparable schools” 
(ibid., 2010b, pp. 20), at least in the case where analysis covers an extended time frame of 
renewed status. 

In the study by Machin and Wilson (2008) reviewed above, the GCSE attainment of all four 
cohorts of Labour’s Academy schools largely consisted of pupils who sat for their GCSE exams 
in the Academy but who entered the school at the beginning of their secondary phase of 

no 

education five years earlier, when the school was in its predecessor years . Thus the estimated 
‘Academy effect’ reflects the outcome of pupil learning in both school types and, importantly, is 
based on a pupil intake that was determined by the predecessor school(s). It is plausible to 
suggest that, once a school converted into an Academy, it faced a strong incentive to make 
compositional changes in the school in order to increase its likelihood of higher GCSE 
performance in the long-run. In particular, gains might have been sought through changes to the 
academic quality and social background composition of pupil intake into the Academy school 
relative to the profile of pupil admissions into its predecessor(s) so that, five years after re- 
opening as an Academy, that more favourable pupil intake yields higher levels of GCSE 
attainment. In this case the ‘Academy effect’, which is entirely attributable to pupil learning in 
the Academy, would appear improved relative to that attached to earlier cohorts admitted by the 
predecessor school(s). This would then boost the chances of the Academies Programme as a 
whole delivering on aim (1) of the policy, where this goal required the schools to contribute to 

~ 8 The GCSE performance of these Academies will also include those pupils who were not in the school (and its 
predecessor) for all five years leading up to the GCSE exam stage. The first cohort of Academies opened in the 
academic year 2002/03 and their GCSE attainment as Academies can be tracked for four years under the sample 
window of the reviewed study, until 2005/06. Pupils who took their GCSEs in the Academy in 2005/06 will, in most 
cases, have entered the school in 2001/02, as a year 7 entrant of the predecessor version of the school. Hence, even 
among the earliest cohort of Academies, the sample window includes the GCSE attainment of pupils who attended 
both versions of the school. 
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driving up standards through increasing levels of achievement among their own pupils. In fact, 
Curtis et al. (2008, pp. 16-17) note that “[o]ne of the intermediate objectives related to. ..[aim 
(l)]...was for Academies to achieve the national average for attainment (at various levels) within 
four years of opening.” If changes in intake ‘quality’ took place immediately after conversion, 
raised pupil performance at the GCSE stage after five years of Academy opening could certainly 
be more easily achieved, resulting in a greater chance of the accomplishment of this intermediate 
aim, albeit with a delay of one year. If this were the case then the findings of Machin and Vernoit 
(2010b) - with initial results showing improved GCSE attainment among Academies as the 
duration of the scheme’s implementation has increased - may be driven by compositional 
changes in these schools. Importantly, employing a strategy of changing the pupil profile within 
an Academy school carries implications for delivery on goal (3) of the Academies Programme, 
suggesting a conflict of interest in the “ultimate” objectives of the scheme. This situation 
provides the key motivation behind the focus of evaluation in this research, the empirical starting 
stages of which are discussed below. 


3 Outlining the Empirical Process 


Academy chools ample 

Prior to describing the data sources that are used for the purposes of empirical evaluation, it is 
necessary to set out details on the sample of Academy schools that the analysis refers to. As was 
noted in the ‘Introduction’ section to this paper, the first cohort of Academy schools came into 
being in September 2002 and additional cohorts arose in each academic year following on from 
then, continuing along the lines of Labour’s Academies model until the Coalition government 
took over in May 2010. At the initial time of writing (June 2009) there were 133 open Academy 
schools dispersed across 65 LEAs, of which a total of 46 (in 34 LEAs) could be traced in the 
available pupil-level and school-level datasets obtainable at the time of data analysis (August 
2008). Table 2 lists each of these 46 Academies that opened between the school years 2002/03 
and 2006/07 and also provides facts on their date of opening, their geographical location, the 
relative deprivation ranking of the area in which each functioning Academy school is situated, the 
amount of finance the sponsor(s) committed themselves to contributing to the school, and the 
subject area(s) of specialism for every Academy. The Table also includes other information 
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relevant for the empirical work, as will be discussed in section 4, ‘Empirical Results: Main 
Findings’, such as whether each established Academy school represents a completely new school 
or simply a new building, the number of predecessor schools that the Academy has replaced, and 
if such replacement has involved a school that was formerly a CTC. 

As can be seen from Table 2, the majority of Academies featuring in the sample period opened 
during the 2006/07 academic year, when a total of 19 were launched, as compared with 3 opening 
in 2002/03, 9 in 2003/04, 5 in 2004/05 and 10 in 2005/06 (column 2). Most of these Academies 
are located in London, in line with the previous government’s goal of establishing 60 Academies 
in this region by 2010. Altogether 23 Academy schools were set up in London in the five years 
since the programme began, corresponding to half of the aggregate amount, of which 13 were 
formed in inner London and 10 in the outer London regions. Following behind Greater London is 
the area of Yorkshire and the Humber, containing a far fewer sum of 6 Academies over the 
sample time-frame. At the Local Authority District (LAD) level, Middlesbrough (in the North 
East) and Southwark (in Inner London) each had three Academies in them by 2006/07, more than 
in any of the other LADs (column 3). These Academies are situated in districts that are 
characterised by high levels of deprivation, ranking 9 th and 26 th respectively (out of 354 LADs) 
on the 2007 Indices of Deprivation . In fact, the vast majority of Academy schools shown in the 
Table have been formed in disadvantaged areas: column 4 reveals that 34 Academies (out of the 
44 with an available deprivation ranking for their area) feature in the 100 most deprived 
localities, based on the 2007 Indices. This conforms with the notion that, in its original form, the 
scheme was to target weak schools in areas of decline, and therefore the underprivileged pupils 
that frequently attend these schools and inhabit such areas . 


Moving on to address Academy school sponsorship, both the United Learning Trust and the 
Harris Federation of South London Schools Trust have gained prevalence in the programme as 
multi-Academy sponsors, the former being involved in whole or in part with nine of the listed 
Academies and the latter with four. Sponsor financial pledges to Academies have averaged £1.69 


29 The 354 district-level authorities comprise 36 metropolitan districts, 32 London boroughs, 284 non-metropolitan 
districts, the Isles of Scilly, and the City of London (see the section on district ‘types’ in particular from 
h ttp ://c ncyc loped i a. thef reed ictionary.com/districts+of+En gland (accessed 3 March 2009)). The Indices of 
Deprivation for 2007 is based on seven domains, namely income deprivation, employment deprivation, health 
deprivation and disability, education, skills and training deprivation, barriers to housing and services, crime, and the 
living environment deprivation (The English Indices of Deprivation, 2007). 

30 According to the DCSF Standards Site the expectation was that by September 2008 around 50% of the 100 most 
deprived Local Authority Districts (LADs) in England would feature at least one Academy school, where deprivation 
is measured according to The English Indices of Deprivation 2004 and concerns a ranking system for all 354 LADs. 
See http://www.standards.dcsf.gov.uk/academies/what are academies/working/?version=l (accessed 21 August 
2008). 
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million, which is about 6.3 to 7.0 per cent of the overall cost of recreating a school into an 
Academy, depending on whether the school was formed through an entirely new build or as a 

o 1 

refurbishment (column 5) . The most frequently chosen subject of specialism has been that of 
Business and Enterprise, either as a sole specialism or in conjunction with another field of study. 
Otherwise, sponsors have tended to opt for sports as their Academy’s area of expertise (column 
6 ). 

Table 2 additionally highlights specifics pertaining to each Academy school, as shown in column 
7. In some LADs (such as Middlesbrough and Southwark) two pre-existing schools were 
amalgamated into one Academy, while unique to the LAD of Westminster has been the 
replacement of one predecessor school by two Academies. A total of five new schools were set 
up as Academies from 2002/03 to 2006/07, raising the number of available schools and therefore 
school places in their respective localities. For seven predecessor schools a change to Academy 
status resulted in capital expenditure on a school rebuild rather than the use of the existing school 
facilities. And finally, five Academy schools had formerly been a CTC, with the largest 
conversion of this kind taking place during 2005/06 when 3 CTCs changed to Academy school 
status. This conversion has been described as natural, given the close connection in design 
between CTCs and Academies (Curtis et al., 2008). It is likely that this would have been a more 
prominent feature of future Academy cohorts developed under Labour, since in early 2009 the 
party had suggested a further extension of the model that involved encouraging successful 

IT 

schools, such as CTCs, to become Academies . 

Though the Academies listed in Table 2 are spread across several LEAs and, in line with 
education policy at that time, their magnitude and dispersion was on the rise, Academy schools 
were not expected to account for a significant fraction of state secondary education provision 
until around 2015, by which time 400 such schools were intended to be in existence (around 15 
per cent of the total: see section 1, ‘Introduction’). Table 3 indicates that by 2006/07 Academy 
schools held just a 1.4 per cent share in the overall stock of state secondary schools. Their 
allocations of pupils and teachers at this time were equally low, at 1.3 per cent and 1.5 per cent 
respectively, while within Academies this slight over-balance of teachers has allowed for a 


31 See section 2, ‘Key Features’ (part (iv)) for the estimated costs of Academy formation according to the NAO. 
There is information available on the committed financial contributions of the sponsor(s) for 43 of the listed 
Academies, totalling £72.55 million, or about £1.69 million on average. 

32 For further details see Curtis et al. (2008), section 4 (pp. 50-67). Government interest in converting all CTCs to 
Academies has been expressed on the following website: 

http://www.standards.dfes. gov. uk/academies/ctcs/?version=l (accessed 20 February 2009). 
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relatively smaller pupil-teacher ratio (15.06) compared to that in all maintained secondary 
schools (16.47). 


Data escription 

Information on all pupils who have been or currently are enrolled in the state schools education 
sector in England is contained within the National Pupil Database (NPD), a central government 
collected longitudinal data source that comprises of the Pupil-Level Annual School Census 
(PLASC) and Key Stage data files. PLASC is a unique national pupil-level administrative Census 
which has traditionally derived information on the whole school roll in January of each academic 
year. The Census has legally binding status: schools are statutorily required to provide Census 
information under Section 537A of the Education Act 1996 (Harland and Stillwell, 2007). 
PLASC contains some indicators on the background characteristics of each pupil, such as 
whether the pupil is eligible for Free School Meals (FSME) , whether the pupil has Special 
Educational Needs (SEN), the ethnicity of the pupil, their gender and their first language. These 
details are provided alongside items such as the school year group to which the pupil belongs, the 
code of the school that they are currently in, and the LEA within which that school is contained. 
PLASC collection first began in January 2002 to include pupils on roll for the academic year 
2001/2002. At the time of carrying out empirical evaluation (August 2008) six PLASC waves of 
data had been issued, covering the academic years 2001/02 to 2006/07 inclusive, all of which are 
used in this empirical work 34 . 

The National Curriculum was established through the 1988 Education Reform Act and provides a 
standard form and content of subjects to be taught across schools for all pupils from the age of 5 
to 16. It was in place in all maintained primary and secondary schools between the academic 
years of 1989/90 and 1996/97. The Curriculum divides schooling years into blocks, with each 
block representing a ‘Key Stage’ (KS). Curriculum comprehension is tested through national 
attainment examinations taken at the end of each KS. Formal introduction to the Key Stages 


33 See Appendix 1 for an explanation of the parental financial or other conditions under which their children are 
entitled to free school meals. This indicator provides a proxy measure for family disadvantage, the drawbacks of 
which are also discussed in Appendix 1. 

34 Since 2006 (2007) a tri-annual procedure for administrative data collection was introduced into secondary 
(primary) schools, known as the School Census and featuring data collection points on the third Thursday of the 
months of May and September in addition to the usual (third Thursday of the month of) January record (Harland and 
Stillwell, 2007). However for researchers the year-on-year January collection is the most longitudinally available and 
consistent source, and therefore the most widely used version. 
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begins at the age of 5/6 (KS1) and comprises of 2 school years of instruction, leading to KS1 
examinations at the age of 6/7. The KS2 phase of learning spans 4 school years and final exams 
are sat for when pupils are aged 10/11. Until recently, after a further 3 academic years, which 
include a transfer from the primary to the secondary schooling phase (at around the age of 11), 

K 

KS3 exams were taken at the age of 13/14 .At the age of 15/16 the end-of-compulsory- 
schooling GCSE exams are taken (KS4). Analysis undertaken here utilises pupil test performance 
at KS1 and KS2, the latter corresponding to the end of the primary school phase of education. 
Information on the code of the school attended by the pupil at the time of their KS3 tests is also 
exploited here. KS1 and KS3 data are provided in the NPD from the academic year 1997/98 
onwards; those for KS2 are available from 1995/96. PLASC and KS records can all be linked 
together by means of a unique, anonymous, pupil identifier provided in every data file. 


Statistics on school-level characteristics are contained within the Edubase, School Performance 
Tables (SPT), and Annual School Census (ASC) data sources, which are collected by the central 
government education department. Edubase is a register of all schools in England and Wales that 
is available from the academic year 1999/00. Details on the number of pupils in the school and 
the school type (such as Community, Independent, etc.) can be obtained from this source. League 
tables of the performance of secondary schools were established since 1994 and contain 
information on the percentage of pupils getting nationally recognised GCSE qualifications at the 
age of 15/16 in each school. The consistent indicators of GCSE attainment that are available in all 
years of SPT data are those of the percentage of pupils attaining five A*-C grades at GCSE and 
the percentage of pupils getting five A*-G GCSE grades at the school-level. Pupils not achieving 
any GCSE passes are those with grades lower than the G level in all subjects; therefore the annual 
percentage of pupils with no GCSE passes can be calculated as 1 minus the percentage of pupils 
getting five A*-G GCSEs. ASC data covers all schools in England and provision of these 
statistics by schools is a legislative requirement of the 1996 Education Act. School-level 
information provided by this source includes the percentage of pupils who are eligible for free 
school meals, the percentage of pupils with special educational needs with and without a 
statement, the percentage representation of different ethnic groups of pupils in the school and the 
pupil-teacher ratio. All annual school-level factors derived from the three data sources outlined 
here are matched to the NPD dataset by the school code. 


35 These have since been abolished with effect from October 2008, such that the last academic year in which they 
were sat for was 2006/07. They have been replaced by teacher assessment. See 

http://www.direct.gov.uk/en/Parents/Schoolslearninganddevelopment/ExamsTestsAndTheCurriculum/DG 1001304 
_1 (accessed 25 February 2011) and http://www.telegraph.co.uk/education/primaryeducation/3199156/Sats-for-14- 
year-olds-abolished-Teachers-and-parents-praise-decision.html (accessed 15 October 2008). 
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Dataset onstruction 


The empirical analysis made in this paper looks at changes in the academic quality and 
composition of pupils entering year 7 of secondary school in each year as well as whole-school 
level year-on-year compositional changes. These angles of enquiry can be assessed using a 
dataset compiled from the above sources, as set out here. 

Changes to intake composition in secondary schools can be examined over the 6 PLASC waves 
only. PLASC provides a sole source of information on the background characteristics of pupils 
joining each school, with the indicators as outlined above being available for each pupil in each 
wave. The variable contained in PLASC on the national curriculum year group to which each 
pupil belongs can be used to identify and extract pupils entering year 7 of each secondary school 
per year from the full PLASC population^ 6 . Of this year group, only those pupils entering 
secondary schools situated in the 34 LEAs in which the sample of 46 established Academy 
schools are situated are kept. This sample restriction is imposed because one purpose of the 
analysis is to define a control group of schools whose intake patterns and changes in school 
composition can be compared with those in existing Academies and their predecessor 
counterparts. If they are to provide an accurate comparison, schools in the control group should 
resemble Academy predecessors by sharing similar characteristics to these schools, but being 
differentiated by the fact that they did not acquire Academy school status. One such attribute is 
the geographical location of comparison schools. Elimination from the sample of those pupils 
entering schools that are not located in an LEA in the vicinity of a formed Academy represents an 
initial movement towards developing an adequate set of comparison schools. 

Changes in the academic intake quality of new secondary school pupils can be assessed by 
linking in KS2 records to the PLASC sample of year 7 pupils using the anonymous pupil 
identifier. KS2 outcomes provide a measure of the academic achievement of each pupil before 


36 As Table 2 showed. Academy schools generally opened in September, and schools traditionally start their new 
school year in this month, while PLASC information on the pupil roll that is used here was collected in January. This 
gap of approximately 4 months in the data collection point does create the potential for a discrepancy to exist 
between the recorded details on pupil enrolment and who actually entered the school. However, it is likely that the 
amount of the discrepancy is too small to have any discernable impact on the findings, and it should be emphasised 
that the unit of analysis in empirical evaluation is the school rather than the pupil. Then the year-on-year variations 
that are witnessed in the data can be considered to be quite accurate, even with pupil entry and exit potentially 
occurring in these 4 months. 
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secondary school entry, so that the social background details of pupils entering secondary schools 
over 2001/02 to 2006/07 are adjoined to the end of primary school prior attainment of these 
pupils over 2000/01 to 2005/06. One way to lengthen the window of information on pupil intake 
quality changes so that the years before PLASC are covered is to exploit details on the secondary 
school attended by each pupil when they sat for their KS3 exams and track this information back 
to establish which pupils entered year 7 of that same school in each year. Information on the KS2 
performance of these pupils can be linked in using the pupil code, as it was for the PLASC year 7 
sample. In this case pupils who took their KS3 exams in year 9 of secondary school at the age of 
13/14 should have entered the first year of that secondary school, year 7, two school years earlier 
when aged 11/12, and should have sat for their KS2 tests in primary school one year before then, 
when aged 10/11. Although pupil-level KS3 attainment data is available in the NPD from 
1997/98 onwards, KS2 data is only provided from 1995/96 and this matches to the 1998/99 KS3 
outcomes of the same pupils. Thus there are no KS2 records that link up to the initial KS3 year, 
making it redundant to the analysis. Then the overall sample can be expanded by at most 5 years 
at the front, to provide 11 years of data on changes in the academic intake quality of pupils 
joining secondary schools, beginning in the academic year 1996/97. 

The assumption that underlies the use of the KS3 data in this way is that pupils did not move 
schools between year 7 and year 9 of secondary school. If pupils who were in the school in year - 7 
left by year 9, then KS3-derived information on the set of pupils who were in the school two 
school years earlier will be smaller than the actual figure. Conversely, if pupils who took their 
KS3 exams in the school were attending a different school in year 7, the sample size will be 
larger than it should be. Mobility of this type will matter for the analysis if pupils exhibiting 
certain characteristics are more likely to engage in moves around this period, a situation which 
will affect the accuracy of empirical estimation. Research has shown that school mobility during 
the secondary phase of education is actually lower than that during the primary phase; 6.4 per 
cent of pupils make non-compulsory changes of school over the entire KS1 period (when aged 
5/6 to 6/7) and 5.0 per cent move schools during KS2 (aged between 7/8 and 10/11), compared 
with mobility of just 3.4 per cent at KS3 (when pupils are secondary school and are aged 1 1/12 to 
13/14) (Machin et al., 2006; Wilson, 2009) 37 . At this point it is worthwhile to note that the 
reliability of estimates obtained using KS3 details to derive year 7 cohorts in years prior to 
PLASC availability is considered in section 4, ‘Robustness Checks’, and it can be stated here that 

37 Non-compulsory school moves refer to those taking place at non-standard times, thus they exclude expected 
transitions such as from Primary to Secondary school. Infant to Junior school, and other forms of necessary school 
changes (see Machin et al., 2006; Wilson, 2009). 
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the KS3-derived part of the sample does appear to act as a valid proxy for determining actual 
pupil entry to each school in the years before PLASC. 

Table 4 highlights the number of years over which the longitudinal panel of observations on 
pupils entering the same group of secondary schools has been created using both the PLASC 
dataset and extrapolated KS3 information. The Table also shows the year-on-year number of 
pupils entering the sample of secondary schools, plus the number and percentage of these pupils 
that have been successfully linked to their previous KS2 attainment records. As can be seen from 
the Table, between around 120,500 and 129,000 pupils joined year 7 of the set of secondary 
schools sampled here in each year. For the majority of these pupils their prior attainment records 
at the end of primary school are available: the match on KS2 test scores lies between 89.6 per 
cent and a very high 97.6 per cent. This provides assurance that intake quality changes can be 
effectively analysed with the information contained in the constructed dataset. 

Implicit throughout the discussion of the sample formation so far has been the notion that the unit 
of analysis is the school rather than the individual pupil. Extracted pupil-level information on 
entry to year 7 of secondary school is cross-sectional in nature and the consistent longitudinal 
component here is the sample of schools these pupils entered into. Though they are pupil-level 
files, both the PLASC and the KS3 parts of the NPD indicate the secondary school to which each 
pupil currently belongs, enabling them to be collapsed in order to generate a sample that is at the 
level of the individual school. In creating the school-level dataset, all characteristics pertaining to 
pupils entering year 7 of secondary school become expressed as fractions, totals or averages at 
the level of each secondary school, depending on the background indicator in question. 

Whole-school level compositional changes can be examined by adding to the dataset indicators 
on the school that are provided in the centrally-collected Edubase and ASC files. These files can 
be linked to the school-level dataset created so far using the school code. The Edubase data 
source is available from the academic year 1999/00 onwards, while ASC data is provided for 

io 

each academic year of the entire sample period spanning 1996/97 to 2006/07 . This step in 
dataset development is an important one for enriching the evaluation that is carried out as it 
allows for a better-defined comparison group of schools to Academy predecessors to be 

38 Whole school-level variables that are linked in from Edubase for the school years 1996/97 to 1998/99 make use of 
the Edubase information for 1999/00. This is a feasible practice because the extracted indicators are relatively time- 
invariant at the level of the school. It should be noted that school codes differ between the predecessor years and the 
Academy years of each Academy school. Linkage of both Edubase and ASC information via school codes is 
therefore done according to the relevant code applying to the school in each year. 
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established, as will be outlined in section 3, ‘Empirical Methodology’. As for intake quality 
changes, whole school compositional changes can be assessed over the 11 year period of 1996/97 
to 2006/07 39 . 

The final stage of dataset construction involves various procedures that are applied to the sample 
of schools in order to arrive at a balanced panel of school-level observations. Creating a balanced 
panel ensures that the findings from analysis into variations in intake patterns and school-level 
compositional changes across schools are not distorted by attrition in specific variables or in an 
entire set of annual observations in the sample of schools. The routes taken to create this final 
dataset are set out in detail in Appendix 2. Table 5 indicates the size of this sample of schools 
before and after corrections and imputations were made. Panel A shows that the sample of 
Academies dropped from 46 to 33 schools, while the total number of all other state schools 
located in an LEA that features at least one Academy school (henceforth termed the sample of 
“non- Academy schools”) fell from 1,699 to 389 schools following the process of data cleaning 40 . 
The entire schools sample is contained within 25 LEAs, rather than 34 LEAs as was originally 
the case, which is a direct consequence of some Academy schools being lost from the sample, an 
issue that is discussed further in Appendix 2. It is worthy to note here a total of 5 Academies were 
dropped because they formed new schools with no historical pre-policy observations and 2 
Academy schools fell from the sample because their predecessors opened later than the start of 
the sample period, of 1997. The difference in the drop in the number of Academies (of 13 in 
total) as compared with the loss of LEAs (9 altogether) reflects the fact that some LEAs contain 
more than one Academy school. 

Panel B of Table 5 shows when the switch to Academy status occurred for each of the 5 cohorts 
of Academies for which empirical details were available at the time of analysis (August 2008), as 
well as how many schools each cohort has contained. As per the initial sample of Academy 
schools (shown in Table 2), in the final sample the largest cohort of Academies are those that 
opened from September 2006, cohort 5. This is also the group from which the most Academies 


39 From here on academic years will be referred to by their end year, such that where 1997 is written in the text, for 
example, this should be interpreted as referring to the academic year 1996/97. 

40 The original number of non-Academy schools in the sample, of 1,699, is inflated by the presence of schools that 
cannot be directly compared with Academies because their institutional arrangements differ (such as independent 
schools) and also by the unusually high number of small schools that are contained in the dataset in 2006. The latter 
likely reflects an error in records that is unique to this year, since across all other years of school-level data assessed 
here (1997-2005, and 2007), there are around 600 non-Academies. These and other errors were corrected for, as 
detailed in Appendix 2. 
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are lost in reaching the balanced panel - seven Academy schools dropped out in this year, as 
compared with none from cohorts 1 and 4 and three each from cohorts 2 and 3 (see Appendix 2). 

Academy school cohorts are divided between their predecessor years (P) and their Academy 
years (A), depending on the timing of their institutional conversion. It is anticipated that this 
break in the status of these schools is marked by a change to their pupil intake patterns and 
whole-school composition; these wzY/im-Academy school policy responses form a further aspect 
of the analytical enquiry to follow (see section 4, ‘Dynamic Effects’). For non- Academy schools 
their status remained unchanged (U) throughout the period, apart from for a negligible percentage 
of schools in this group (see Appendix 7 for further details). 

To summarise, the nature of the constructed dataset allows for changes to intake composition 
among secondary schools to be analysed over the 6 year window of 2002 to 2007, while intake 
quality and whole school compositional patterns can be examined for 11 years encompassing 
1997 to 2007. The amount of predecessor school information is therefore lower when intake 
composition effects are addressed, since these details are only available in PLASC, and neither 
KS3 extrapolation nor the use of school-level Edubase or LEASIS/ASC files can be used to 
extend the window of this pupil-level dataset. 


Empirical ethodology 

The main methodological approach adopted here is that of a ‘difference-in-differences’ analysis 
applied to the constructed longitudinal dataset which contains school-level factors that are tracked 
through time. This technique involves comparing the difference in an outcome measure in the 
‘treated’ group of schools (those that switched to Academy status) with that in an appropriate 
comparison group in the years before and after the school status change was implemented. 
Observed changes reflect the pre-and post-policy excess in the average of the outcome measure 
in the treatment group vis-a-vis the comparison group. Estimation produces a parameter that 
identifies the average impact of treatment on the treated, or the ATT (Blundell and Costa Dias, 
2008). Equation (1) below indicates the basic difference-in-differences model that is applied to 
the sample of schools here and the key coefficient of interest that derives from model estimation. 
The relationship between an outcome measure y in secondary school s in a certain time period t 
and model covariates can be specified in an equation as:- 
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}’„ =^ + P\ + SA s * Policy On t > k + £ v , 
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The term A in this equation refers to the ‘Academy’ dummy variable. This takes the value of 1 for 
schools that became Academies, and covers all 11 years of the school (their pre-policy 
predecessor school years and their post-policy Academy school years); otherwise the variable 
assumes the value of 0 across all 1 1 years in non- Academies. The constant or intercept is denoted 
by a and e is the error term, a variable that incorporates all unobservable components that are 
associated with the particular outcome measure. The main parameter of interest is S on the 
A s *PolicyOn t >k variable. PolicyOn t >k is the treatment variable, a dummy indicator that equates to 
1 over the time periods ( t > k ) in which the Academy school policy is in effect in school s and 0 at 
all other times (so that k is the year that a school re-opened as an Academy). The coefficient S 
captures the average change in the outcome measure within the treated group of schools relative 
to the comparison group, after the school status change occurred, and is therefore an estimate of 
the ATT parameter. Throughout the analysis that follows the coefficient expression ‘S\ and the 
terms ‘ATT’, and ‘academy on’ will be used interchangeably to all refer to this estimated 
treatment impact. 

A more detailed model specification is set out in equation (2) below, which includes regressors 
that additionally account for observable attributes of schools that may relate to the outcome 
indicator. Explanatory variables that further exploit the nature of the fixed-effects method in 
being able to control for unobservable time-invariant factors that may impact on outcomes 
directly or via correlation with assignment to the treatment group are also modelled here 
(Emmerson et al., 2003) 41 :- 


y« = * PolicyOn til + VZ rl + £ +A,+ e st ( 2 ) 


41 Unobservable factors consist of time-constant and time-varying components. The difference-in-differences method 
accounts for the impact of time-constant unobservable effects on the outcome measure. Time-varying unobservable 
effects could take the form of (i) an unexpected one-off event, e.g. a sudden change in the composition of a 
neighbourhood, which affects Academies simultaneously opening in that area at the time of its occurrence; or (ii) a 
change that occurs through time, e.g. the process of neighbourhood gentrification, which will impact on the 
neighbourhood composition and on Academies within the area over time, and therefore will display a time-trend. The 
impact of random events such as case (i) cannot be netted out using the difference-in-differences approach, a 
limitation of the method that is likely to be small given the unlikelihood of these events happening. The effect of case 
(ii) can be modelled by fitting a time-trend to the data over all available years and estimating whether the policy 
effect is attributable to patterns, or ‘trends’, that were already present in the outcome measure over the pre-policy 
period. This exercise is carried out as part of a series of ‘robustness checks’ of empirical findings on changes to 
intake ability in Academies (see section 4 for the results of these checks). 
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In the above equation Z represents a vector of observable school-level characteristics, with 
associated coefficients Y. The term refers to a set of year dummies that are incorporated in the 
model so as to net out unobservable year-specific effects that are common to all schools in each 
year (and differ across years). & indicates a set of school dummies that are added to the 
difference-in-differences regression in order to account for time-constant observable and 
unobservable characteristics that are unique to the individual school. That is, the term controls 
for the impact of school fixed effects on y st . In this case all observable features of schools that are 
unchanging over time become absorbed in the school fixed effect term, including the Academy 
dummy variable (the J3A S part of equation (1) above). The regression now models the within- 
school effects of Academy status on each outcome measure . 

Defining a suitable comparison group of untreated schools constitutes an important part of the 
process of empirical evaluation. This set of schools provides the closest possible counterfactual 
scenario, illustrating patterns of behaviour that might have existed in Academy schools had they 
not participated in Labour’s policy of status change. So far a sample of non-Academy schools has 
been established for this purpose, where this group contains only those state-maintained schools 
of the traditional type that feature in an LEA in which at least one Academy school came into 
being over the time period considered (see Appendix 2 for further details). While these untreated 
schools may represent an adequately defined control group, reaching a well-defined set of non- 
Academies enables more accurate sample estimation of the ATT parameter, bringing that 
estimation closer to the true value. Better definition can be achieved by reducing the 
heterogeneity between the characteristics of non-Academy schools and those of Academy pre- 
policy predecessor schools as much as possible, such that Academies and non-Academies share a 
similar probability of being subjected to the policy treatment based on their attributes and only 
differ according to their actual treatment status. Resemblance in the pre-policy characteristics of 
the two groups of schools matters because it is on factors such as these that the status change is 
likely to have been based. 


42 More specifically, the regression with school-fixed effects models deviations from school-specific means. Thus 
deviations of the dependent and independent variables for each school from the school-specific average of these 
variables over the time period concerned are estimated. In this case any time-constant terms in the regression 
equation that involve grouped schools are no longer separately identified since they become subsumed within the 
school fixed effect. As the model with school dummies provides estimation at the lowest hierarchical unit, that of the 
individual school, it gives a much more unique and informative ATT coefficient than models estimated at more 
aggregated levels. 
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Of course the heterogeneity that is present between Academies and untreated schools reflects 
both observable and unobservable dimensions, and the dataset used here provides information on 
schools that allows for only certain observable differences to be taken into account. Even if data 
pertaining to every aspect of schools were collected and freely available, the selection rules 
governing assignment to the original Academies programme were not precisely stated, making 
the task of netting out heterogeneous differences less clear. In general Academy school 
‘treatment’ under the Labour government depended on the partially observable features of 
schools that concern their performance and their levels of disadvantage (see section 2, ‘Aims and 
Objectives’). The National Challenge definition of an underachieving school (as one where 30 
per cent of pupils or more do not attain five good GCSEs in the A*-C range, including in English 
and maths) was used as one qualifying criteria for school replacement by an Academy by the 
previous government from 2008. In terms of the data sources used here, the percentage of pupils 
not getting any GCSE passes gives an indicator of poor school performance, while a crude 
measure of school-level disadvantage is provided by the percentage of pupils eligible for free 
school meals in the school. Though they are incomplete determinants of eligibility for Academy 
treatment, the availability of statistics on these observable treatment participation components 
allows for some of the variation between the treated and untreated schools to be separated out. 
Therefore some control schools that do not have observable historical attributes resembling those 
of Academy predecessors can be excluded from the analysis. In fact, the advantage of the 
constructed dataset is that it contains school-level details stretching as far back as 1997 and 
incorporates available information in the year just prior to the decision of each Academy school 
to convert to Academy status. Then historical and more recent trends in these observable factors 
that likely influenced assignment to treatment among schools can be put to use as a means for 
strengthening the analysis findings 43 . 

The procedure that is employed in order to determine a distinct control group of schools is that of 
estimating a statistical propensity score for each school and then restricting the entire sample of 
schools to those contained within a common support region under which only Academy and non- 
Academy schools with similar propensity scores feature. The propensity score for a school is the 
[0, 1] conditional predicted probability of assignment to the treatment group for that school, that 
is, the likelihood of a school becoming an Academy given the available set of pre -policy 
observable factors relating to it. This conditional assignment probability can be estimated in a 
parametric non-linear logit or probit model or through a linear probability model, where the 

43 In fact, a whole host of school-level observable variables are tested for their ability to predict assignment to the 
Academy treatment group, as will become clear in the discussion that follows. 
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parametric specification expresses a relationship between the actual treatment status of the school 
and their observable pre -policy variables. Hence the parametric equation models the Academy 
dummy variable given by A s in equation (1) on the left hand side and all pre-policy observable 
covariates of schools considered to have mattered for determining assignment to Academy status 
‘treatment’ on the right hand side. The coefficients derived under the process of parametric 
estimation are used to predict a propensity score for each school. The area of overlap in the 
distribution of the propensity scores of the treatment and control groups indicates those Academy 
and non- Academy schools sharing similar treatment probabilities, and is known as the ‘Common 
Support Region’ (CSR). Schools that are excluded from this region are those displaying a very 
different set of observable characteristics, such that their likelihood of becoming an Academy, as 
summarised in their propensity score, is either above or below the threshold points of common 
support 44 . All empirical results shown in this paper pertain to the sample of schools within the 
CSR and the sensitivity of findings to the relaxation of this constraint is included as a category of 
robustness checking. It should be noted that the construction of a reduced sample of schools - 
produced by the estimation of propensity scores and the associated derivation of a CSR - is 
carried out as a preliminary stage to the analysis and acts as a subsidiary to the main method of 
empirical difference-in-differences regression estimation, under which the treatment impacts 
themselves are gauged 45 . 

The process leading up to the generation of the CSR sample begins with the presentation of 
descriptive statistics on the entire sample of schools prior to CSR formation, followed by the 
tracking of how disparities in the characteristics of treated and control schools are narrowed down 
once restriction to the CSR occurs. In Panel A of Table 6, indicators on the composition of 
Academy predecessors and all non-Academy schools are presented in the form of school-level 
averages covering the pre-policy window that is common to all Academy cohorts contained 
within the sample, 1997 to 2002. These descriptive variables illustrate statistically significant 
differences in the pre-treatment observables of Academy predecessors and the full control group 
of schools. In line with the tendency of Labour’s Academies to be set up in areas of decline, the 


44 In practice both treatment and control schools may be discarded from the empirical analysis if their propensity 
scores do not fall within the common support region. It will be seen from the logit models presented in Table 7 (see 
also Figure 1 and Figure 2) that none of the 33 Academy schools are excluded from the difference-in-differences 
evaluation process since each of their propensity scores are featuring in this region of overlap. 

44 As a precursory stage to the regression analysis the propensity score (and subsequent common support) approach 
has the major advantage of being able to make use of all observable school-level characteristics for which data is 
available, while not all of these can be included in the difference-in-differences equations as independent variables 
on the right hand side because many constitute the left hand side outcome measures. Thus the combination of this 
initial step and difference-in-differencing means that as many observable and unobservable dimensions of schools as 
possible are controlled for. 
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Table shows that their predecessor versions were characterised by a far higher proportion of 
pupils eligible for free school meals than is the case in non- Academy schools, where this measure 
is a proxy for family disadvantage (see Appendix 1). Over the 6 year period just above 40 per 
cent of pupils were entitled to FSM on average in the pre-Academies, as compared with about 25 
per cent in the whole non- Academy group. 

Schools with poor attainment standards were those most likely to convert to Academy status and 
the tabulated statistics reveal that this holds in the schools sampled here. On average almost one- 
quarter of the predecessor school population completed their compulsory schooling years 
achieving no GCSE qualifications (22.25 per cent), while about the same percentage acquired 
five or more GCSEs graded A*-C (25.45 per cent). Non- Academy schools fared better all round, 
with just 12.46 per cent of pupils not gaining any GCSE passes across all 6 years on average and 
38.34 per cent acquiring the nationally recognised standard of achievement at the GCSE stage. 
This latter percentage of 38.34 is important as it crudely indicates that non-Academies achieved a 
sufficiently high enough level of GCSE performance to sit outside of the definition of an under- 
achieving school ripe for conversion to an Academy that was determined within Labour 
education policy from 2008. Of course, this recent definition would in no way have influenced 
conversion to Academy status among the schools featuring in Table 6. Additionally, included in 
this qualifying criterion for conversion is a focus on attainment in English and maths in 
particular, while an historical breakdown of per subject GCSE attainment at the school-level is 
not available in the utilised data sources. 

Panel B of Table 6 shows the average characteristics of primary schools at the time when they 
were attended by pupils subsequently entering year 7 of the secondary schools sample in each 
pre-policy year. It appears that the compositional differences between Academy predecessors and 
non-Academies stem in part from compositional variations in the primary schools from which 
these secondary schools got their pupil intake 46 . Indeed pupils joining predecessor secondary 
schools over 1997 to 2002 tended to come from primary schools with higher levels of social 
disadvantage. The percentage of pupils eligible for FSM in the primary schools from which 
predecessor schools sampled is 39.14 per cent, as compared with 26.83 per cent in the primary 
schools that non-Academies sampled from, a statistically significant difference of 12.31 
percentage points. Interestingly, pupils entering pre- Academy schools were apt to come from a 
larger number of lesser-performing primary schools. Predecessor schools spread their year 7 

46 The reader should note that the statistics in Panel A of Table 6 are at the whole school-level; they do not indicate 
school-level averages of pupils entering year 7 only. 
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intake over 36 primary schools on average with a mean KS2 primary school performance of 
71.00 points. This compares with non- Academies sampling their year 7 intake from 34 primary 
schools averaging a higher KS2 quality of 74.56 points. The Labour government set a target of 
attainment at KS2 of Level 4 in each of the three tested subjects of English, maths and science, 
the points score equivalent of which is 81 (27 points in each subject). Though school-level 
averages mask individual variation, it is likely that more pupils entering non- Academies achieved 
the target level of KS2 attainment in all subjects than did pupils who were admitted into 
Academy predecessor schools. 

The pre-policy observable characteristics of Academy and non-Academy secondary schools 
shown in panel A of Table 6 are mapped into implied probabilities of each school becoming an 
Academy using the non-linear logit models as set out in Table 7. The distribution of propensity 
scores obtained from a logit specification fits well to this sample of schools in particular as the 
logit function displays wider tails and a smaller central distribution than does the probit function 
as an alternative model. Therefore the logit model is better able to estimate implied propensities 
in the extremes of the [0, 1] space for a given set of observable characteristics, areas around 
which the predicted probabilities of non-Academies (close to zero) and Academies (close to one) 
can be expected to lie. Although it was highlighted in Table 5 that the cohorts of Academy 
schools contained within the sample have been set up in different time periods - so that the 
Academies differ by their predecessor and policy on years - logit estimation undertaken here is 
based on averaged variables across the 6 pre-policy years (1997 to 2002) that are shared by all of 
these Academy cohorts. This process of defining a single pre-policy period into which all 
Academy predecessors are grouped results in the identification of a single common support 
region and one control group of non-Academies that acts as the counterfactual for all Academy 
schools. Given that some cohorts of established Academy schools are very small in size, 
derivation of a cohort-by-cohort common support region and control set of schools (where 
variations in pre-Academy and Academy policy on years are taken into account) can add little to 
the process of estimation of treatment effects. Hence throughout the empirical analysis that 
follows, testing uses the restricted sample of schools contained within this single CSR and 
involves a comparison of intake behaviour changes and changes in whole school composition 
within all Academies and separate Academy cohorts relative to the unique group of non- 
Academy schools. 
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Columns (1) and (2) of Table 7 are based on estimation of a fully-specified logit model (model 
1), in which all the observable pre-treatment factors in Panel A of Table 6 are used as regressors. 
The results from this model suggest that averaged school-level variables on the fraction of pupils 
in the school with Special Educational Needs with a statement, the pupil-teacher ratio and the 
percentage of pupils getting no GCSE qualifications are good predictors of the likelihood of 
school conversion to Academy status according to their percentage effects. However only the last 
of these variables retains any statistical significance in the estimation process and otherwise all 
other explanatory components are redundant to the analysis. Model 2 of Table 7 represents a 
more parsimonious version of the full logit model, in which indicators that could be 
endogenously determined by the school (SEN status) or that are highly correlated with another 
covariate (the percentage of pupils gaining five or more grade A*-C GCSEs) are excluded from 
the equation 47 . Once again the only statistically significant independent variable is the percentage 
of pupils with no GCSE passes at the age of 15/16 48 . 

The implied probabilities of school change to the Academy type that are derived from the logit 
model with full controls (model 1) display a distribution as shown in Figure 1. The common 
support region pertaining to this model includes the full sample of Academy schools (33) but a 
smaller number of non-Academies (266 out of 389), so that 123 non-Academy schools are 
discarded from the comparison group. A similar graphical interpretation of the region of common 
support derived from the propensity scores achieved under estimation of logit model 2 is given by 
Figure 2. While this area of overlap also includes all 33 Academies, fewer non-Academies are 
excluded from the region than was the case for the CSR associated with model 1. A total of 63 
non-Academy schools drop out of the counterfactual set, leaving 326 control schools that share 
similar pre-treatment observable features to Academy predecessors over the 1997 to 2002 
window. Despite the relatively weak explanatory power of these pre-policy observables in 
determining whether a school was to become an Academy, the subsequent process of defining a 
CSR does generate more stringent testing by reducing heterogeneity in the characteristics of 


47 See Table A2 in Appendix 3, which shows the correlation coefficients among all pre -policy school-level variables 
averaged over 1997 to 2002. The coefficient of correlation between the percentage of pupils gaining five or more 
grade A*-C GCSEs and the percentage of pupils getting no passes at the GCSE stage is a statistically significant - 
0.8023. This very high inverse relationship between these two indicators suggests that at least one of them should be 
excluded from the logit model, as their informative content is the same. The former indicator was chosen to be 
dropped because poor school performance, which is signalled through variables such as the percentage of pupils 
attaining no GCSE qualifications, is one important dimension of school conversion to an Academy according to the 
model of the scheme developed under Labour. 

48 Various other logit model specifications were tested for their predictive capabilities, and none were found to 
improve on the predictive power of the models presented here (see Appendix 4). 
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treatment and control groups 49 . It is the restricted sample of schools contained within the CSR 
linked to logit model 2 on which difference-in-differences regression estimation is to be based 
overall. The logit model with selected controls is marginally better able to predict schools that 
remained as non- Academies (93.11 per cent correctly predicted, as shown in column (3) of Table 
7) than the logit model with full controls (92.97 per cent, column (1)). Logit model 1 can instead 
better identify future Academy schools (98.35 per cent correctly predicted compared with a 
slightly smaller 97.80 per cent under logit model 2). Given that neither of the CSRs originating 
from logit models 1 or 2 exclude any Academy schools, it would appear that the somewhat 
stronger predictive capabilities of logit model 2 in relation to the non-Academies sample 
constitutes sufficient justification for the use of schools in the CSR relating to it. Thus regression 
estimation covers all Academy schools and a wider and more flexible comparison group of non- 
Academy schools than would be the case were the CSR of the full logit model used 50 . 

It is worthwhile to briefly highlight the value in the finding that the CSRs pertaining to logit 
models 1 and 2 both include the full set of 33 Academy schools. Accurate definition of the 
propensity scores used to define these CSRs requires that all factors affecting assignment to the 
treatment group are known, can be observed, and that data on these factors are available to the 
researcher. Inaccuracies in treatment probabilities will therefore reflect unobservable components 
and/or unavailable data on variables that may have contributed to determining treatment 
assignment. In the present case, the fact that propensity scores correctly predict actual Academy 
school status among all Academies featuring in the sample therefore suggests these probabilities 
are well-defined by the set of observable characteristics on schools that are available in the 
dataset. 


4 Empirical Results: Main Findings 


The outcome measure that most illustrates the extent to which schools that switched to Academy 
status became more inclusive and mixed ability - therefore having the potential to enhance 


49 More specifically, the statistically significant differences in the pre-treatment attributes of Academy and control 
schools shown in Panel A of Table 6 are reduced in the formation of a common support region under both logit 
model 1 and the logit specification with selected controls (see Appendix 6, including Tables A4 and A5). 

50 The ‘Empirical Results: Main Findings’ section (4) that follows includes as a robustness check the sensitivity of 
difference-in-differences estimation to variations in the CSR, where one such variation is to use the CSR established 
under the fully-specified logit model. It will be evident from this analysis that the sample ATT parameter is 
unaffected by changes to the CSR (see Table 10). 
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equality in educational opportunity - is that of the average KS2 performance of pupils entering 
year 7 of all sampled secondary schools in each year. As an indicator of the prior attainment of 
pupils joining the school, this outcome measure might be expected to be inversely correlated with 
Academy school conversion, given that Academy schools developed by the Labour government 
tended to be set up in areas of disadvantage often characterised by pupils with low levels of 
academic achievement. In the available data, changes to intake quality in predecessor versus 
Academy schools and in control schools compared to ‘treated’ Academies can be gauged from 
information on the average KS2 total points score of pupils joining each school across the 11 
years of 1997 to 2007. This then forms the dependent variable y st shown in equations (1) and (2) 
of section 3, ‘Empirical Methodology’. 

Table 8 shows the evolution of the average value of this indicator in each year for the 33 
Academy schools overall, separate cohorts of Academies and the restricted control group of non- 
Academy schools contained within the CSR identified from logit model 2 in Table 7. According 
to each category of schools the Table also indicates the change in the outcome measure between 
the initial year (1997) and most recent year (2007) for which data was available at the time of 
empirical assessment (August 2008; see column 12). The difference-in-differences estimates of 
this change in the outcome measure between the first and last year are highlighted in column 13 
of the Table. Here the progression in school-level KS2 intake quality in both grouped Academies 
and each Academy cohort is compared to that in the restricted counterfactual group of 326 non- 
Academy schools within the CSR. The estimation equation is given by:- 

y„ = A + A * PolicyOn,, k +/t, + £„ (3) 

That is, the outcome measure is regressed against the academy dummy variable A s (with 
associated coefficient /?), an interaction term that distinguishes the Academy years from the 
predecessor years in each Academy school over the sample window (i.e. PolicyOn equals 1 in 
those years at and following the policy of conversion ( t > k ) in Academy school s), and a set of 
year dummies which control for within-year effects that are common to all schools and are 
denoted by X,. 

In line with the notion that original Academy schools would frequently feature pupils with a 
relatively weaker background of educational achievement, Table 8 shows that the KS2 total 
points scores of pupils entering grouped Academies (which includes their predecessor 
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counterparts) were consistently below those for pupils joining the sample of non-Academy 
schools in each of the 1 1 years shown. Although the gap in the outcome measure between these 
two groups of schools narrowed over the period, by 2007 Academies still sat below non- 
Academies in their intake quality distribution. However, Academy schools as a whole 
experienced a sharper rise in their pupil intake quality across 1997 to 2007 than did non- 
Academies. Column 12 of the Table indicates that the prior attainment of year 7 pupils jumped up 
by 15.95 KS2 total points in all 33 Academies combined between the end-points of 1997 and 
2007 as compared with an increase of 13.56 in the outcome measure among the restricted sample 
of control schools. The difference-in-differences estimates of column 13 reveal the relative 
change between the treated and comparison group to be statistically significant, with an estimated 
S coefficient of 2.38 on the interaction expression of equation (3). When estimation used the full 
sample of 389 non-Academy schools, the outcome measure changed by 13.08 KS2 total points, 
from 66.97 in 1997 to 80.05 in 2007. The 8 parameter increased to 2.87 (with a standard error of 
0.89) in this case (note that these results are not reported in Table 8). Therefore the process of 
restricting the estimation sample to those Academy and non-Academy schools within the CSR 
results in a more precise and conservative estimation of the relative change in pupil intake quality 
because observable heterogeneity between the two groups of schools is reduced in this region. 

Looking at individual cohorts of Academies, the prior academic performance of pupil entrants 
went up the most amid those schools opening under the new status in the school years 2002/03 
(cohort 1) and 2004/05 (cohort 3), with their KS2 total points rising by 16.63 and 17.52 
respectively. Changes in the outcome measure among these cohorts seem to be the main drivers 
of the grouped change, given that statistical significance only holds for their estimated 
coefficients on the interaction term. It is worthwhile to point out here that caution should be 
exercised in the reading of these findings. Cohort-by-cohort estimates in general, and those 
already mentioned in particular, have been based on a very small number of Academies (cohort 1 
featured only three Academy schools; cohort 3 contained just two Academies). Thus difference- 
in-differences estimation that uses these small sample sizes possesses little informative statistical 
content as compared with results that pertain to the larger sample of grouped Academies. 

Academy schools that opened in 2005/06 (cohort 4) stand out as the group for whom average 
KS2 intake quality was high throughout the 11 year period and in most years (except 1999 and 
2002) this lay above that in non-Academy schools. As was noted in section 3, ‘Academy Schools 
Sample’ (see also Table 2) it was in this year that Academy school conversion was undertaken by 
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several former CTCs. The CTC scheme, as a forerunner to the Academies programme, involved 
much the same process of replacing underachieving schools in disadvantaged areas with 
refreshed set-ups specialising in technology that were independent of LEA control and were 
sponsored by private business. By 1994 this initiative reached its peak with a total of just 15 
CTCs formed, half the original anticipated amount. CTCs are often reported to have out- 
performed other schools within their areas in terms of the number of pupils getting GCSE passes 
in the A*-C range (Astle and Ryan, 2008) 51 . While raised attainment may be a product of 
improved standards of teaching and learning in CTCs, the evidence presented here also points 
towards a more favourable policy of admissions into these schools of pupils with a stronger 
ability background, the upshot of which may be higher school performance in the long-run. 
Overall, as an initial step in the analysis of intake quality changes in Labour’s Academies, the 
results of Table 8 suggest that these schools admitted a different quality of year 7 pupils once 
they switched status relative to both their predecessor school(s) and the non- Academies. It would 
appeal - that in general students of a higher academic ability have been more likely to enter into 
the renewed school. 


In Table 9, the findings established so far from simple difference-in-differences regression 
estimation that used data from the end years of the sample period only are subjected to further 
testing. Here information contained in all 1 1 years of the sample frame is fully exploited, while 
stringent testing based on the restricted sample of schools contained within the CSR is upheld. In 
the first two columns of Table 9 the Academy dummy variable of equation (1) is broken down 
into cohort dummies that distinguish and group Academy schools by their academic year of 
opening. In this case the estimation equation becomes:- 


y„ = 


2>a 


+ <St, * PolicyOn lik +A,+ (O sj + £ sl 


(4) 


c - 1 


Where c ranges from 1 to 5 depending on the cohort to which the academy school belongs; is 
the set of year dummies; and co S j are a set of LEA dummies, one for each of the 25 LEAs in the 
sample. These are included so as to capture unobservable factors that are specific to each LEA (j) 


51 In 2007 CTCs averaged 91% of pupils gaining 5 GCSEs in the A*-C range, compared to a 60% average among 
comprehensive state schools. Including the subjects of English and maths in this category, CTC performance 
dropped to 70%, though this was still much higher than that in other state secondary schools (45%) (Astle and Ryan, 
2008). 
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and affect all schools (5) within the respective LEA in the same way over time . With cohort 
dummies added to the regression equation the coefficient S gives the average change in the 
dependent variable when the effective policy on period is allowed to vary by the Academy 
cohort. 

As can be seen from columns (1) and (2) of Table 9, Academy schools and their predecessors 
have tended to intake year 7 pupils of a lower prior ability than non- Academy schools. Findings 
from the estimation of equation (4) show that almost all of the coefficients on the cohort dummies 
are negative and statistically significant, whether controls are added for year dummies only 
(column 1), or both year dummies and LEA dummies (column 2). The exception is the fourth 
Academy cohort, whose average KS2 intake quality over the 1 1 year period was above that of all 
other non- Academy schools in the sampled LEAs (though this difference is not significant). 
Estimation of how intake quality changed in schools once the Academy policy came into effect 
reveals there to be a sharp jump up in the outcome variable in the conversion years. The 
statistically significant and positive S coefficient indicates that when schools switched to 
Academy status their KS2 total points score was on average 2.460 points higher than in their 
predecessor years and compared to non- Academy schools. Benchmarking this against the sample 
average value of the dependent variable in the pre-policy year that is common to all Academy 
cohorts of 2002, the interpretation of this result is that the average KS2 total points score 
increased from 73.577 to 76.037 when schools re-opened as Academies, a rise of some 3.34 per 


Further disaggregation of the cohort-by-cohort analysis to the level of the individual school is 
enabled through the inclusion of controls for school fixed effects and the results deriving from 
this estimation method are shown in the final two columns of Table 9. In this case the more 
detailed specification of equation (2) is followed (see section 3, ‘Empirical Methodology’). 
Column 3 of Table 9 excludes the vector of observable school-level characteristics that are 
present in equation (2) from regression estimation, while column 4 takes these into account. 


52 With LEA dummies modelled, regression analysis estimates deviations of the dependent and independent variables 
for each school from the LEA-specific average of these variables across all schools in the LEA over the entire time 
period (see column (2) of Table 9). This represents a higher level of aggregation than when school fixed effects are 
added (columns (3) and (4) of Table 9), in which case the regression models deviations from school-specific means. 

53 This is an approximate effect since schools converted into Academies in different years; therefore there is variation 
in the actual final pre-policy year applying to each cohort and the 2002 benchmark value represents the true final 
predecessor school year for the first cohort of Academies only. Taking into account the differing final pre-policy year 
mean values of the outcome measure by cohorts just changes the level at which the average KS2 total points score 
sits for each cohort following their conversion to an Academy, but the end result that there is an average jump up in 
KS2 intake quality across all Academy cohorts still remains. 
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Assessment of the within-school effect of Academy status on KS2 intake quality reveals a largely 
unchanged sample ATT parameter from that estimated at the cohort level; the S coefficient 
remains statistically significant throughout and is only marginally reduced by the inclusion of 
observable school-level controls, falling from 2.460 to 2.409. That this finding remains even after 
controlling for the size of the school (in terms of the numbers of pupils it contains) is significant, 
as it suggests that the result is not explained away by the potentially larger pupil capacity of 
Academy schools, as might have been expected. Schools that became Academies did not simply 
increase their admissions of pupils with a stronger ability background whilst maintaining constant 
intake numbers of pupils from the rest of the ability distribution as before. Instead the results 
found here are indicative of changes to the pupil profile in Academy schools, such that the entry 
of higher ability pupils to these schools was made possible by changes in the distribution of 
intake patterns elsewhere. Likewise even after consideration for the capability of Academies to 
have a lower pupil teacher ratio through their freedom to offer reward schemes that could attract 
more teachers to the school, the substantial increase in the prior attainment of year 7 entrants 
holds 54 . With the coefficient (standard error) on the school size standing at 0.002 (0.001) and that 
on the pupil-teacher ratio being 0.002 (0.032), only the first of these variables is statistically 
significant but neither of them add enough explanatory power to the estimation equation to 
change the end result 55 . 

Overall these regression findings tally with those from the descriptive analysis of Table 8, and 
reaffirm that Academy schools sat below non-Academies in their intake quality distribution 
throughout the 1 1 year period (except for the fourth Academy cohort) but there was a significant 
narrowing of the gap in the outcome measure between these two groups of schools. This is 
particularly evident in the effective years of Academy school status, as the more rigorous 
regression testing presented in Table 9 has now shown. 


54 The notion that Academy schools might be able to accommodate a larger pupil capacity than their predecessor 
version(s) is suggested in section 2, ‘Aims and Objectives’, and the flexibility that Academy schools have had to set 
their own pay and conditions and to offer reward packages to teachers according to aspects such as their performance 
is discussed in section 2, ‘Key Features’ (in particular see point (vi) on staffing). 

55 These coefficient estimates are not reported in Table 9. Further school-level controls for the percentage of pupils 
getting 5+ GCSEs in the A*-C range and for the percentage of pupils without any GCSE passes were added to 
estimation equation (2), both separately and together. Including the former variable reduced the Academy on effect 
(standard error) from 2.409 (0.575) to 2.249 (0.573), while including the latter variable reduced the policy on effect 
to 2.307 (0.563). Including both variables, the S coefficient fell to 2.208 (0.563). In all cases the statistical 
significance of this coefficient estimate remained. This suggests that their inclusion adds little to the findings and, 
given that these GCSE performance indicators refer to a different cohort of pupils from those entering year 7 of the 
school, they have been omitted from further analysis where a vector of observable school-level controls is used. 
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Robustness hecks 


The existence of an Academy effect on KS2 intake quality found in the regression analysis 
discussed above may be due to the nature of the sample restrictions and methodological 
approaches applied to the sample of schools, or due to some as yet unaccounted for pre-policy 
differential trends in this outcome measure across treatment and control schools. In order to 
establish whether the impact on the outcome measure of school conversion into an Academy has 
been correctly identified various robustness checks are carried out and the findings from this 
process are presented in Table 10 56 . The specific equation on which robustness tests are 
conducted is that which delivers the most conservative estimate of the Academy effect, namely 
equation (2) (see section 3, ‘Empirical Methodology’), where the regression results relating to 
this model are given in column 4 of Table 9. If rigorous testing leaves these results unaffected, 
then this gives assurance that the analytical procedure utilised here identifies the effect of the 
policy. 

To begin with, column 1 of Table 10 explores the notion that the positive S coefficient is biased 
downwards by the presence of CTCs that converted to Academies in the sample of Academy 
schools. The CTC programme of school conversion into an LEA-independent technology focused 
institution was developed in England in the late 1980s and early 1990s, and culminated in the 
creation of 15 CTCs by 1994. As an early take on what has since constituted the Academies 
programme, the Conservative government’s CTC scheme and the schools that it established were 
to increasingly become an integral part of the Labour government’s Academy schools agenda. A 
total of five CTCs changed to Academies between the school years 2002/03 and 2006/07. Of 
these, four remain in the restricted set of 33 Academy schools for which all observable data over 
the 1 1 year sample period is available, three of which switched to Academy status during 2005/06 
(cohort 4). The discussion surrounding Table 8 drew attention to the relatively greater KS2 intake 
quality of this cohort, a pattern that is evident across 1997-2007, indicating that predecessor 
CTCs were already admitting pupils of a higher prior ability than other pre-Academy schools. 
This observation points towards the potential underestimation of the Academy effect achieved so 
far due to the sample incorporation of CTCs-turned- Academies; for these schools the change in 


56 Unless otherwise stated all robustness tests are based on Academy and non-Academy schools contained within the 
common support region determined by the logit regression as defined in Table 7, model 2 (see also Figure 2). 
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the outcome measure between predecessor and Academy school years appears to have been lower 

C7 

than that among other Academy schools . 

Removing former CTCs and their respective Academies from the common support region sample 
of schools reduces the set of Academies from 33 to 29. Two such schools represent the only 
Academies in their separate LEAs over the sample period. Once they are dropped all other non- 
Academy schools also featuring in these LEAs and forming part of the control group 
subsequently become redundant to the analysis. Thus the sample of non-Academies falls from 
326 to 294 schools (a loss of 34 schools) following this adjustment. Re-estimation of equation (2) 
on the smaller set of Academy and non- Academy schools produces a larger status change effect; 
the 5 coefficient increases from 2.409 to 3.046 KS2 total points scores. While the larger sample 
size relating to the initial coefficient estimate is favoured over that with sample exclusions, this 
exercise has shown that the impact on the outcome measure of school renewal is actually stronger 
than that first estimated when CTCs that became Academies are left out of the sample. Overall, 
the move of the coefficient in the anticipated direction following sample redefinition of this ki nd 
indicates that the findings from evaluation achieved up to this stage are being consistently 
estimated throughout. 

In columns 2 to 4 of Table 10 the main regression result is tested for sensitivity to changes in the 
groups of treatment and control schools, or, more precisely, consideration is made here for the 
impact on estimation of changes to the common support region from which these groups derive. 
Thus the target of these tests is to understand whether the obtained coefficient relies heavily on 
the structure of the particular sample of schools on which it is estimated. In column 2 of the 
Table, equation (2) is applied to the complete sample of Academy and non-Academy schools 
with a full set of observations in all 11 years of data, and not just to those schools falling within 
the overlapping region of common support. This is equivalent to removing from the 
methodological approach the procedure used to progress from an adequately determined to a 
well-defined comparison group of schools that was set out in section 3, ‘Empirical Methodology’. 


57 The average KS2 total points score of pupils entering CTCs during the pre -policy period that is common to all 
Academy cohorts (1997 to 2002) is 83.917, and in the Academy school years of these CTCs post-conversion (2003 
to 2007) this average increases to 88.374, a rise of 5.312%. Among other Academy schools, their predecessor years 
average is 67.635 and this increases to 74.402 during the Academy years, a gain of 10.005%. Hence this reveals a 
potentially higher KS2 total points score level in CTCs compared to other Academy predecessors that is followed by 
a lower change in this dependent variable once CTCs converted into Academy schools relative to once other 
predecessor schools made the change. 
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As can be seen from the Table, relaxing this sample restriction leaves the set of Academy schools 
unaltered and increases the set of non- Academies by 63 schools, to 389 schools. The end results 
that this produces on the ATT parameter are to maintain its statistical significance and to increase 
its estimated size by 0.138 KS2 total points scores, from 2.409 to 2.547, so that there is an 
increase in the Academy effect implied by the differences in these two coefficients of 5.74 per 

CO 

cent . That conditioning estimation to the sample of schools within the CSR generates a lower 8 
coefficient is a finding which is in line with expectations. The process of identifying a common 
support region aims to improve the precision with which the counterfactual scenario is defined, 
leading to the sample elimination of non-Academy schools differing greatly in their observable 
pre-policy characteristics (and hence their implied propensity of treatment) from Academies. The 
design of a more stringent sample frame that ensues delivers more conservative policy effect 
estimates because heterogeneity between treatment and control schools, in terms of variation in 
their observable attributes, is reduced by this method. Thus the outcome of this initial sensitivity 
analysis is in accordance with the main regression result. 

The resilience of the Academy impact to variations in the common support region is tested in 
columns 3 and 4 of Table 10. Column 3 uses the propensity scores and CSR pertaining to the 
logit model with full controls, shown as model 1 in Table 7 (see also Figure 1). In column 4 the 
likelihood of conversion to Academy status for each school and the CSR are re-estimated using a 
non-linear probit model on the same set of selected controls as for logit model 2 in Table 7 59 . In 
both of these cases the overlapping region of common support includes fewer non-Academy 
schools than does the CSR associated with logit model 2, the preferred logit specification, while 
all 33 Academy schools remain. For the logit regression with full controls the CSR is smaller by 
61 non- Academies, 60 of which feature in the CSR determined under logit model 2, and one of 
which does not. The probit model that uses selected controls is smaller by 57 non-Academy 


58 This percentage increase is determined as follows: (2. 409/73. 577)*100 = 3.274%; (2. 547/73. 577)*100 = 3.462%; 
and ((3.462-3.274/3. 274)*100) = 5.74%. Here the value 73.577 is the average KS2 total points score in 2002, the 
common pre -policy year for all Academy cohorts (see the notes to Table 9); 3.274% is the percentage change in this 
average when the common support restriction is applied to the sample; 3.462% is the equivalent percentage change 
in this average when the common support restriction is dropped; and therefore 5.74% gives the percentage increase 
in the Academy effect as a consequence of the difference in the two estimated coefficients. 

59 The probit model, like logit model 2, only estimates a statistically significant marginal effect on the percentage of 
pupils getting no passes at the GCSE stage. The percentage effect contribution of this observable variable on 
predicting the probability of a school becoming an Academy is 9.42% using logit model 2; in the probit model the 
equivalent percentage effect is higher, at 12.06%. The logit and probit models are equally good at correctly 
predicting which schools remain as non-Academies (93.11% are correctly predicted under both models), but the 
probit model is marginally better at predicting which schools become Academies (98.35% versus 97.80% under logit 
model 2 - see also Table 7). In this respect, the results from probit model estimation support the notion of the relative 
importance of poor school performance in determining school conversion into an Academy, thus being in accordance 
with the original model of the scheme as was developed by the former Labour government. 
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schools, all of which are contained within the CSR of logit model 2. The distribution of the 
propensity scores derived under the probit model is illustrated in Figure 3. 

Interestingly, the reduction in the number of schools in the comparison group of non- Academies 
that results from common support area changes makes little difference to the size of the estimated 
Academy effect and leaves the statistical significance of this effect unchanged. The use of a fully- 
specified logit model cuts the S coefficient by just 0.009 KS2 total points scores. This suggests 
that employing a more parsimoniously expressed logit model that consumes less degrees of 
freedom by requiring the coefficients on fewer explanatory variables to be evaluated represents 
an effective technique. The S coefficient relating to the probit model is smaller by 0.05 1 points, at 
2.358. It was noted in section 3, ‘Empirical Methodology’, that the logit model produces a 
distribution of implied probabilities that exhibits wider tails than the probit model, so that the 
former non-linear specification is better able to estimate extreme propensity scores on the edges 
of the [0, 1] space. This aspect of the logit model makes it better suited to the schools sample 
used here, given the clear division in assignment to treatment status for Academy schools versus 
non- Academies. The logit model is more likely to group the probability of assignment to the 
treatment group around one for Academy schools and close to zero for non-Academies than the 
probit model, which instead generates a larger central distribution of treatment propensities. The 
fact that the logit regression identified more non- Academy schools in the CSR than did the probit, 
even if modelling used the same set of selected observable controls, provides evidence of the 
relatively stronger capabilities of the logit model in predicting extreme probability values, and 
hence the better application of this non-linear form to the current dataset. Overall the sensitivity 
tests carried out in columns 3 and 4 of Table 10 indicate that the estimated Academy effect is not 
responsive to variations in either the specification or the functional form of the non-linear model 
used, nor to the resultant changes in the CSR that re-estimation of propensity scores produces. 
Given that a smaller sample of comparison schools is contained in both of the alternative non- 
linear expressions, the preferred logit model has the comparative advantage of allowing 
estimation to utilise a greater number of observational units. 

The final three columns of Table 10 assess whether the witnessed Academy effect is attributable 
to the nature of the trends that the outcome measure was following in schools in the years prior to 
Academy status introduction. Column 5 looks for differential trends in KS2 intake quality 
between Academy and non-Academy schools in the pre-policy period that continue into the 
effective years of the Academy policy and that can account entirely for the estimated ATT 
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coefficient. The difference-in-differences regression models the policy impact assuming that a 
discemable gap in the outcome measure between Academy and non- Academy schools displays a 
common and parallel trend across all 1 1 years of data. In the effectual policy years the expression 
(SA s *PolicyOn,>ic) in the difference-in-differences equation allows for the size of this gap to 
change, but the parallelism of the outstanding distance in KS2 intake quality between treated and 
control schools is assumed to remain. If instead there is evidence of differential trends in the 
outcome measure between the two groups, then the estimated S coefficient may just be capturing 
these. Hence this part of the robustness analysis amounts to an explicit test of the validity of the 
common trends assumption on which identification of the ATT parameter using the difference-in- 
differences estimation procedure relies. If the common trends assumption does not hold then this 
introduces bias into the ATT parameter so that the difference-in-differences method does not 
consistently estimate the ATT coefficient (Blundell and Costa Dias, 2008) 60 . Differential trends 
can be accounted for by including in equation (2) additional controls that interact the individual 
school fixed effects (modelled as school dummy variables 61 ) with a term that counts the school 
years (m), where m equals 1 to 11 for each year of data (1997 to 2007) pertaining to school s. The 
regression equation then becomes:- 


y„ = <&« * Policy On,^ + TZ„ + £ + 


359 




s* m s 


• + A + £ 
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As shown in column 5 of Table 10, the § coefficient is robust to the inclusion of school-specific 
trends in the difference-in-differences regression; the Academy effect is positive (at 2.136 KS2 
total points scores) and statistically significant. This means that there is no evidence of 
differential trends in the outcome measure between Academy and non- Academy schools that can 
account for the policy effect. The common trends assumption is not violated here and as a 
consequence the sample ATT is consistently estimated using the difference-in-differences 
approach. 


In columns 6 and 7 of the Table a falsification exercise takes place that involves testing the 
robustness of the main regression result to the notion of trends in the outcome measure in treated 


60 Consistency is a large sample property. The sample ATT parameter will be a consistent estimator if in the limit of 
the sample size (that is, when the sample size increases indefinitely) the distribution of this estimator collapses to a 
single point (with zero variance around that point) that represents the true ATT value (Gujarati, 1995). 

61 Notice that coefficients on a total of 359 school dummies interacted with the school year count ( m ) are estimated, 
corresponding to the sum of 326 non-Academies and 33 Academies contained within the common support region. 
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and control schools exhibiting a similar historical pattern in the years prior to the Academies 
scheme as that displayed by the two groups in the pre-post policy period. If there is evidence of 
an analogous evolution in the dependent variable occurring at some previous time interval, then 
the jump up in KS2 intake quality that is attributed to the impact of school change to Academy 
status simply reflects unaccounted for pre-existing variations in the outcome measure between 
treated and control schools. An effective way to assess whether this is the case is to run an 
experiment where, for each Academy cohort, their total number of years of school status as an 
Academy are shifted to an earlier time period. If re-estimation of the Academy effect in this 
‘fake’ policy set-up gives a similar result to that in the true policy framework then the measured 
impact is fully accounted for by historical trends. In this test the policy period should be moved 
so that the alternative Academy school era does not overlap with the real policy on phase of any 
of the cohorts. If overlap does happen then the fake experiment may contain treatment 
contamination in the explanatory variables. Also, the experimental scenario should include 
observational points in which schools were not Academies, to allow a pre-versus-post policy 
evaluation to take place. Given the abundance of historical information on schools contained 
within the dataset used here, there are enough years of data available to make this testing method 
viable. In particular, the six years 1997 to 2002 represent a universal pre-policy period across all 
sampled Academy cohorts to which the experimental setting can be applied. Table 11 indicates 
the practicalities behind this testing process and how the ‘fake’ trial situation compares with that 
which has actually existed for each wave of Academy schools. 

As can be seen from the Table, the fake policy experiment does not use data from any years 
corresponding to those where schools had converted to Academies (2003 to 2007). Instead 
difference-in-differences estimation is based on the six pre-policy years that are common to all 
Academy waves. In this respect the experimental setting uses a reduced number of predecessor 
annual observations for each Academy cohort with respect to previously. Two issues relating to 
this change in the sample frame warrant discussion here. The first is whether the actual Academy 
effect that was estimated on the full set of 1 1 years of data is also evident if only 6 years of data 
are utilised. The second is whether this Academy effect exists if the pattern of predecessor and 
Academy school observations on which it is based is made to resemble that in the falsification 
exercise. If these conditions are satisfied then the outcome of the fake test can be directly 
compared to the actual case, since the difference in the number of years used in each regression 
and the change in the pre-post policy set-up do not affect the estimated result of the policy. 
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The outlined sections of the ‘actual’ rows in Table 11 illustrate how the pre-post policy pattern 
and number of annual observations in the experimental test can be mirrored in the actual 
situation. It can be seen that the years 1997 to 2001 are no longer drawn on in this shortened 
sample period. Column 6 of Table 10 shows what happens to the sample ATT parameter in the 
true policy period when it is re-estimated using just 6 annual observations covering 2002 to 2007. 
This reveals there to be a positive and statistically significant change in KS2 intake quality 
among Academy schools even when a reduced number of years of data are employed (8 = 2.388). 
This means that the outcomes from the falsification experiment and the actual result are 
comparable. Column 7 of Table 10 presents the change in the dependent variable arising from 
school conversion into an Academy when consideration is made for a similar evolution in trends 
in this indicator between treatment and control schools in an earlier time period. The finding from 
this falsification test is that the Academy effect is not evident in the pre-policy interval; the S 
coefficient stands at a small and statistically insignificant 0.148 KS2 total points scores in the 
experimental scenario. Therefore the rise in pupil intake quality in Academy schools relative to 
both their predecessors and non- Academies that is found in the actual policy setting reflects a 
genuine impact of school conversion into an Academy rather than a repeat of historical patterns. 

To summarise, all of the robustness checks carried out in Table 10 provide qualification for the 
correct identification of the impact of Academy school status on patterns of intake ability, where 
the measured effect is indicated in column 4 of Table 9. The outcome of re-estimating this effect 
using only 6 years of data (as discussed above and shown in Table 10, column 6) also provides 
assurance about one particular aspect of the evaluation process. It was noted in section 3, ‘Dataset 
Construction’, that pupils joining year 7 of Academy schools, their predecessors and other non- 
Academy schools (featuring in the same LEAs as Academies) in each year from 2002 to 2007 
could be identified using PLASC data that has been available annually from 2002 onwards. The 
academic quality of these pupils could be established from their record of prior attainment in KS2 
exams taken at the end of the primary school stage, and these were linked to the PLASC data 
using a pupil identifier. Over these 6 years a complete record of the academic ability of pupils 
entering the sample of secondary schools could thus be determined from the available PLASC 
data. In order to establish which pupils were entering this set of secondary schools in the years 
prior to PLASC a process of extrapolation that exploited KS3 pupil-level records was employed, 
as discussed in section 3, ‘Dataset Construction’. More specifically, the code of the school 
attended by the pupil at KS3 (when pupils are aged 13/14) was used to infer which pupils were in 
the schools two academic years earlier as new entrants (aged 11/12), and KS2 records of the prior 
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attainment of these pupils were then linked in. This method of extrapolation enabled the sample 
window to be lengthened from the 6 PLASC years of data covering 2002 to 2007 to an eleven 
year period, that of 1997 to 2007. 

One concern about this extrapolation procedure was the potential for pupil mobility between the 
start of secondary school and the time when KS3 exams were taken to generate inaccuracies in 
the inferred records of pupil entry into the secondary schools of interest. A plausible way to 
check whether pupil mobility is an issue is to compare difference-in-differences regression 
estimation of the Academy effect when all 1 1 years of the data are used with that derived from a 
sample window based around only the 6 years of PLASC data (2002 to 2007). The outcome of 
the latter regression using the 6 PLASC years is exactly that shown in the robustness exercise of 
column 6 in Table 10, and, as was discussed, this yields only a fractional downwards change in 
the estimated policy effect in comparison to regression analysis that exploits the full sample 
period. Hence estimation that uses all eleven data points is reliable according to the checking 
approach carried out here. Pupil mobility does not appear to have impacted on records of intake 
into secondary schools in the years 1997 to 2001 and the KS3-derived part of the sample acts as a 
valid proxy for actual pupil entry into each secondary school in the years before PLASC. 

A further point that warrants discussion at this stage is the potential for changes in intake 
behaviour among Academy schools to have affected the intake patterns of other non- Academies, 
so that the measured Academy effect stems from the use of an inappropriate comparison group. 
In particular, if Academy schools competed with non-Academies for pupil intake from the same 
supply pool, then the increased entry of more academically able pupils into Academies may have 
come at the expense of a reduced quantity of this pupil type for non-Academy schools to admit. 
In this case, the introduction of the Academies programme in an area would have resulted in a 
‘crowding out’ effect in the pupil admissions supply for other local schools. Then this raises the 
issue of the validity of using similar schools in the LEA that did not become Academies as a 
comparison group, given that they may not have been unaffected by the programme. 

There are two lines of argument to suggest that policy spillovers are not a major cause for 
concern in the present scenario. Firstly, of the 25 LEAs sampled here, there is on average one 
Academy school featuring in a single LEA, with the highest number being three. In terms of the 
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control group of non- Academies, the mean number per LEA is 13 schools 62 . Given these 
statistics, it is unlikely that intake behaviour changes in one post-conversion Academy school 
could have had an impact on pupil admissions in all 13 control schools within the LEA of that 
Academy. Therefore any contamination effects of treatment on the untreated group are likely to 
be too minor to cause concern. Secondly, the average annual KS2 total points scores of pupils 
entering year 7 of the set of non-Academies were reported in Table 8, where it was shown that 
the comparison group still experienced intake quality growth between 1997 and 2007 but to a 
lesser degree than did all predecessor-turned- Academies. It can be seen from inspection of the 
figures shown in this Table that year-on-year changes in intake quality in the non- Academy group 
are always positive, even in the period of conversion into Academy status by other schools (2003 
to 2007). The fact that pupils entering non-Academy schools were of an increasingly stronger 
academic quality throughout the 11 year window implies that intake into these schools was not 
substantially altered by the presence of competing Academies in the local area. Taken together, 
the arguments raised here provide justification for the use of non-Academies as an effective 
control group, given that there appear to have been no significant indications of policy spillovers 
occurring from Academy to non-Academy schools that might have affected pupil admissions for 
both parties. 


62 Inspection of the data revealed there to be on average one Academy school within an individual LEA, and only 2 
LEAs feature a maximum total of 3 Academy schools. The per-LEA control school averages reported in the text are 
calculated by dividing the total number of non-Academy schools within the CSR (326) by the total number of 25 
LEAs. Without the common support region restriction applied, there are on average 16 non-Academy control schools 
within the LEA of one Academy school (389 non-Academy schools are present in the full sample). In fact, the 
original dataset contained even more non-Academy schools relative to the sample of Academies, some of which 
were dropped in the process of deriving a balanced panel of school-level observations (see Appendix 2, including 
Table Al). Thus all figures discussed here understate the actual number of control schools within the LEA of an 
Academy. 
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Dynamic ffects 


School conversion to Academy status has thus far been shown to be generally characterised by 
these schools having admitted year 7 pupils with a higher record of prior attainment. The 
stringent model estimated in column 4 of Table 9 revealed a statistically significant 2.409 rise in 
the KS2 total points scores of pupils entering Academy schools, a finding that stands up to a 
whole host of robustness checks. This policy effect estimate is assumed to be unchanging over 
time in the Academy years and it indicates the average instantaneous response of all 33 
Academy schools to treatment. In Table 12 tests are carried out which look for evidence of 
dynamic reactions to the Academy schools policy, such that the outcome measure may have 
continued to change among Academies as the length of exposure to the scheme increased with 
time. Testing also asks whether any dynamic or otherwise static effects are coupled with 
heterogeneous impacts of school conversion by each Academy cohort. Here the aim is to 
understand if the estimated ATT coefficient is attributable to the policy responses of a particular 
cohort or cohorts of Academy schools. 

In the first column of Table 12 all schools that became Academies are assumed to have had the 
same initial change in their KS2 intake quality on average (as per column 4 of Table 9), but this 
immediate policy reaction is additionally tested for further changes over time. A non-flat slope in 
the effective policy years would suggest that gradual year-on-year changes in intake quality 
occurred and these added to the instantaneous post-conversion rise. Column 1 of the Table 
preliminarily models this growth rate as being identical for all Academy schools, such that 
estimation follows equation (2) but with a control for time effects inserted as follows:- 

y st = §A s * Policy On t > k + ij\A s * Policy On *(t-k + 1)] + 

X *' Z st + £s + ^t +£ st ( 6 ) 

Here the coefficient ;/ measures the average change in the outcome measure for each incremental 
year of Academy school status ( t-k+1 , where t is the year and k is the year in which the school 
became an Academy). The results of regression estimation suggest that accounting for more 
adjustments in KS2 intake quality as schools continued their Academy experience does little to 
change the average treatment effect, with the S coefficient remaining statistically significant and 
just above 2 KS2 total points scores, at 2.009. This unchanging result arises because there is no 
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significant time-on effect (rj = 0.208; standard error on rj = 0.240), with the implication being that 
the null hypothesis of an initial average rise followed by flat growth in the dependent variable as 
Academy school exposure carried on cannot be rejected. Thus it would appear that there is only a 
one-off augmentation in pupil quality that happened as soon as schools re-opened as Academies. 

The notion that different cohorts of Academies may exert varying degrees of influence on the S 
coefficient is considered in column 2 of Table 12, where this average initial policy response is 
allowed to differ by the cohort. In this case some Academy waves may have changed the intake 
quality of their new entrants by more than others once they became Academies, so that they have 
driven the immediate jump up in the outcome measure. Estimation of the following equation 
establishes a separate S sample parameter for each Academy cohort, c, where c ranges from 1 to 
5:- 


y„ = 


YjS c A sc * PolicyOn, 

l C=1 


t>k 


+ +g s +A t + £ st 


(7) 


As can be seen from the findings in column 2, Academy cohorts responded in much the same 
way to the policy, with a per cohort effect ranging between positive KS2 total points score 
changes of 1.596 (cohort 4) and 3.701 (cohort 3). While the former estimate is not statistically 
significant, there is sufficient overlap in the confidence intervals on these sample coefficients to 
suggest similarity in cohort reactions to Academy status on KS2 intake quality changes. This 
cohort-common policy consequence is formally checked through F-tests that set two separate null 
hypotheses, one of a zero effect on the outcome measure from conversion to an Academy, where 
this non-effect is equal for all Academy cohorts ( S c = 0 for all c), and the other of a cohort equal 
effect (Si = (h = Sj = 64 = S 5 ). These F-tests reconfirm the findings that there was a positive, non- 
zero equal initial change in the outcome measure among the five Academy groups, suggesting 
that no Academy cohort or cohorts in particular generated the average response over and above 
others (see the rows in italics in Table 12, where the p-value on the first F-test in column 2 is 
0.000 so that the null hypothesis is rejected and that on the second F-test is 0.824, so that the null 
cannot be rejected). 

Taking the analysis of column 2 to the next level, column 3 of Table 12 combines heterogeneous 
policy impacts with a non-flat growth in the outcome measure that is common to all Academy 
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schools during the post-policy period. Thus the testing procedures of columns 1 and 2 are 
combined and jointly assessed in this evaluation stage, so that the estimation equation becomes:- 


y„ = 


Yj S cK* Policy On tik 


c = 1 


+ T]\A s * Policy On *(t-k + 1)]+ 


vz, +€s + A + s st 


(8) 


In line with the findings from the earlier tests, there is no evidence of either prolonged responses 
to treatment among all Academy cohorts or differential impacts of status change on the dependent 
variable by the Academy cohort. The // coefficient is not statistically significant and the F- 
statistics suggest a rejection of the null hypothesis that S c = 0 for all c, instead indicating that d, > 
0 and is equal for the sample of 33 Academies. 

In the final column of Table 12 the most flexible pattern of responses to Academy conversion is 
considered, in which all possible facets of cohort heterogeneity are allowed to occur; Academy 
cohorts are tested for differential initial changes in the outcome measure upon switching to 
Academy status as well as for further policy reactions through time that likewise vary by the 
cohort. Then equation (8) shown above is adapted slightly and is written as:- 


y st = 


c = 1 


Z S Ac * PolicyOn ak + Z^ \A SC * Policy On *(?-& + !)] 
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X ^ Z st + £s +Z t + £ st 


(9) 


It should be emphasised that estimation of all of the parameters in equation (9) using a sample of 
just 33 Academy schools is a very demanding exercise. Nevertheless, regression findings reveal 
that the results pertaining to all other columns of the Table remain; there has been a homogenous 
cohort response to the Academy policy and no changes to the outcome measure after the average 
positive change which happened initially when schools converted to Academies. Conclusions 
deriving from the F-tests on the S coefficients of column 3 are unchanged when extended model 
specification (9) is estimated in column 4. Further tests for joint significance of the ;/ coefficients 
across Academy waves indicate that time-on effects take the value of zero and are equal in all 
five cohorts; the p-value on HO: r\ c - 0 for all c is 0.227 so that the null hypothesis is not rejected 
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and that on HO: /// = rj 2 = f ]3 = fj 4 = is 0.438, again suggesting that the null hypothesis holds 
(see the last two rows of Table 12). 

Overall, the dynamic equation models (6) to (9) estimated in Table 12 give weight to the 
persistent regression finding that there was an immediate increase in KS2 intake quality once 
schools became Academies. Beyond this initial rise there have been no more changes in this 
outcome measure over the sample period, though the jump up is a constant impact that has 
neither reversed nor reduced over time in the effective policy years. Moreover, this pattern of 
behaviour has not been witnessed among non-Academy schools, nor was it evident at some 
earlier point in time, as the pre-policy robustness test results in column 7 of Table 10 showed. 
Thus the estimated Academy conversion effect shown in column 4 of Table 9 and derived using 
regression equation (2), which excludes cohort- specific controls, binds in all tested 
circumstances. All Academy schools changed their intake of pupils to those with an average 
2.409 higher KS2 total points score as soon as they opened as Academies, and this sample ATT 
coefficient is statistically significant and robustly identified. The lack of evidence of dynamic 
effects may reflect the small size of the Academy schools sample utilised here. A useful future 
research exercise would be to determine whether heterogeneous cohort responses to the policy 
can be found in a larger sample. This could be assessed using data spanning the academic years 
1996/1997 to 2009/2010 and involving Academies that were set up according to Labour’s model 
of the scheme - that is, encompassing those Academies which were established before the 
Coalition government’s change in the Academy policy upon their coming to power in May 2010. 


Possible echanisms behind hanging ntake uality 

In the final part of the evaluation process to be concerned with KS2 intake quality changes, Table 
13 presents findings from analysis that looks into the mechanisms behind the positive jump in 
this dependent variable among schools that became Academies as compared to other schools in 
the sample. To start with, in the first four columns of Table 13 regression estimation exploits 
available information on the pre-KS2 performance of pupils. Data on the KS1 test outcomes of 
the sample of year 7 pupils were linked in to the pupil-level file already containing their KS2 
attainment scores prior to the collapsing of the pupil-level dataset to the level of the individual 
school. KS1 tests scores are available in the NPD from the academic year 1997/98 onwards; KS1 
tests are taken in primary school when pupils are aged 6/7, and are followed up by KS2 tests 
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which are taken four school years later in the final year of primary school when pupils reach the 
age of 10/11. 

Further details on the historical academic ability of pupils entering the secondary schools sample 
are added in because much can be learnt about the types of higher ability pupils that entered 
Academy schools from their KS1 records combined with their KS2 outcomes. It may be, for 
example, that Academy schools have admitted pupils showing signs of improved learning over 
time, so that their value-added test score gains between KS1 and KS2 are high. Or pupil intake 
into Academy schools may have comprised of pupils showing consistently strong levels of 
attainment, in which case their KS1 and KS2 total points scores may be high but remained at a 
similar level between the two tests so that value-added gains are low. If intake patterns are more 
reflective of stronger growth in value-added then this suggests that Academy schools have 
preferred to admit pupils who attended primary schools that are more likely to have been 
effective in raising educational performance and attainment. On the other hand, higher KS levels 
(and lower value-added increases) among new entrants to Academies implies that admissions 
have been geared more towards pupils with a higher ‘innate ability’ . This is true if early measures 
of attainment, such as KS1 test score outcomes, are perceived to capture pre-determined learning 
capacity that derives from factors like the influence of family background on the pupil rather than 
academic skills acquired in the immediate years of exposure to primary school education. Hence 
further analysis of this kind helps in understanding more about the nature of the KS2 intake 
changes shown to have taken place among Academy schools. 

The addition of KS1 records of attainment to the dataset reduces the number of pupils in each 
secondary school in the sample because details on both KS1 and KS2 performance are not 
available for every pupil that entered these schools 63 . Records of the year-on-year school-level 
average KS2 total points scores are consequently raised or lowered, depending on the 
implications that missing combined KS1-KS2 data has on changes to the pupil intake sample for 
secondary schools. Also, the number of annual observations on the secondary schools sample is 
reduced because KS1 data records existing from 1998 link to KS2 records from 2002, and pupils 
who took their KS2 tests in this year began secondary school in 2003. The starting point of KS1 
to KS2 analysis is therefore cut to 2003 among all secondary schools and the years 1997 to 2002 
can no longer be exploited for their pre-policy informative content on historical intake patterns in 

63 Table At in Appendix 2 shows the annual drop in the sample of year 7 pupils when records on both KS1 and KS2 
attainment are required, as well as the percentage of the year 7 sample with a matching KS 1 record in each year, over 
the period 2003 to 2007. 
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these schools. This means that for the first cohort of Academy schools opening from September 
2002 and completing their first academic year in 2003, there is no KS1 data available to match to 
the KS2 outcomes of pupils entering their pre-policy, predecessor schools in the years before 
2003. This makes pre-post difference-in-differences analysis infeasible for the initial Academy 
cohort and for this reason the three Academy schools in the cohort are dropped from the sample. 
All other non-Academy schools featuring in the LEA of a dropped Academy school are also 
excluded so long as that Academy school represents the only one in the LEA within the utilised 
sample. 

Columns 1 and 2 of Table 13 check the sensitivity of the estimated impact of Academy school 
conversion on KS2 intake quality to changes in the sample structure associated with the use of 
KS1 data matched to KS2 records. In column 1 the trimmed nature of the changed sample 
structure is imposed on the original schools sample. Hence this testing procedure amounts to re- 
estimating equation (2) using the original form of school-level annual average KS2 total points 
scores (in which not all pupils have a matching KS1 record), a reduced number of years (2003 to 
2007), and a smaller set of Academy and non-Academy schools (cohort 1 Academies and 
associated control schools for sole Academies in the LEA of this cohort are dropped). The S 
coefficient falls marginally from 2.409 to 2.339 KS2 total points scores in this case, so that the 
smaller sample frame has little effect on the estimated policy impact. 

In column 2 of the Table, equation (2) is again estimated on the reduced sample structure that 
uses fewer annual observations and secondary schools, but this time all pupil entrants are 
required to have a full record of KS1 and KS2 outcomes. Thus here the dependent variable has 
differing average annual values from the original case depending on how many pupils had to be 
dropped from each school in the sample because they lacked both KS1 and KS2 records. As can 
be seen from the Table, the Academy effect estimate is lower when this sample is used: S drops 
by 0.651 KS2 total points scores to 1.758. So the dependent variable has changed to an extent 
where the estimated policy impact has fallen by more than when the number of years and schools 
on which estimation is based are reduced (as can be seen by comparing the coefficient results 
shown in columns 1 and 2 and the Academy school dependent variable averages in the pre-policy 
year of 2003). It is likely that some of this reduction in the estimated effect stems from 
conditioning the sample of pupils within schools to have both KS1 and KS2 outcomes. Pupils of 
this kind may be of stronger academic ability, to the extent that a regular record of attainment 
indicates greater motivation and commitment to learning. The KS1-KS2 sample also excludes 
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recent immigrants who, by definition, do not have a continuous record of education in the 
country, and who may account for a large share of the lower levels of KS2 attainment. Indeed, 
higher standards of attainment in the group of pupils with KS1 and KS2 outcomes are evident 
from the higher level of pre-policy KS2 intake quality in this sample, resulting in a decrease in 
the measured policy impact 64 . Although the coefficient estimate is lower in column 2, a positive 
and statistically significant jump up in KS2 intake quality among Academy schools remains the 
dominant finding, suggesting that, in general, this result is not sensitive to sample structure 
alterations. 

Having tested whether the estimated Academy effect persists following sample changes, the next 
2 columns of Table 13 use the matched KS1 and KS2 sample to consider if schools admitted 
more of a particular pupil type once they became Academies: either pupils with a stronger innate 
ability background or improved learners who likely attended more effective primary schools. 
Regressing school-level annual average KS 1 levels in the first case and school-level value-added 
in the second case on the right hand side components of equation (2) produces S estimates as 
shown in columns (3) and (4) respectively. The findings suggest that Academy school admissions 
steered marginally towards the direction of inherent pupil academic ability over and above the 
incorporation of pupils with strong value-added gains between the key stages. While the S 
coefficient on KS1 to KS2 value-added is positive but not statistically significant, that on KS1 
levels is higher and has statistical significance at the 10 per cent level (t = 1.70, compared with a 
t - value of 1.645 at the 10 per cent level of significance). This is not a result that stands out and it 
is important to note that, generally-speaking, it is not clear how well informed state secondary 
schools are about the prior attainment of pupils applying for (year 7) entry to their school. 
Nevertheless, this finding has raised the issue of a potential change to the types of pupils 
comprising Academy school intake relative to what went before. 

Exploration of the processes governing intake quality changes in Academy schools now moves 
on to look at variations surrounding the sources of pupil intake into these schools. In column 5 of 
Table 13, consideration is made for whether the number of primary schools from which 
secondary schools got their pupil admissions differs among schools that converted to Academy 
status versus non- Academies. Then column 6 asks if schools that received Academy ‘treatment’ 
subsequently took their intake of pupils from relatively higher performing primary schools than 
did their predecessors or the comparison group of schools. These issues are examined by re- 

64 The mean of the dependent variable for Academy schools in the pre-policy year of 2003 is 75.01 KS2 total points 
under column (2) of Table 13, which is greater than that under column (1), of 73.69 points. 
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estimating equation (2) using as a dependent variable the number of intake primary schools or the 
average annual KS2 performance of these intake primary schools respectively. The results shown 
in column 5 indicate that Academies featuring in the sample increased their primary school 
supply pool following their status switch relative to control schools 65 . The size of the d 
coefficient in this case is estimated as a statistically significant 4.427 intake primary schools. 
Thus the mean number of primaries from which predecessor Academies got their intake in the 
common pre-policy year of 2002 is 33 schools and after conversion this rose to almost 38 
schools, a gain of 13.42 per cent. This increase is found even though the regression equation 
includes a control for the potentially larger pupil capacity of each Academy school compared to 
their predecessor(s) 66 , therefore it is not simply a reflection of school size changes. Turning now 
to primary school performance, the findings in column 6 of the Table show that those pupils 
entering Academies also came from academically stronger primary schools. The average school- 
level KS2 performance of intake primaries is 0.865 total points higher in the Academy school 
years, suggesting that while predecessor schools got their intake from primary schools with an 
average performance of 75.62 KS2 total point scores (in 2002), the quality of primary schools 
from which admissions came once these schools switched to Academies is about 1.2 per cent 
higher, at 76.49 total points. 

In columns 7 and 8 of Table 13 the auxiliary informative content provided by the above lines of 
enquiry into changes in the sources of pupil entry is explicitly modelled in the main difference-in- 
differences regression (equation (2)) with average annual KS2 total points scores as the outcome 
measure. Column 7 highlights what happens to the estimated S coefficient when the indicators 
used to measure these issues are included as supplementary explanatory variables in the 
regression and column 8 adds to this further controls for observable primary school-level 
characteristics, listed in the notes to the Table. 

As can be seen from the results presented in Table 13, these extra regressors do help in explaining 
which factors shaped the rise in pupil intake quality among schools that converted into 
Academies. About 34.70 per cent of the measured boost in KS2 intake ability can be accounted 


65 The number of primary schools from which secondary schools got their year 7 intake in each year is determined 
using information on the code of the school attended by each intake pupil at the time that they took their KS2 tests 
(where these are taken in the last year of primary school). Each different primary school code was assigned the value 
of 1 and values were then summed at the secondary school level. The average annual performance of the primary 
school attended by each pupil entering year 7 of secondary school was averaged again at the secondary school-level 
in order to establish the mean quality of intake primary schools. 

66 Note that school size changes in the sample of secondary schools are captured in the vector of observable school- 
level controls that are expressed in equation (2) by the term l /'Z„. 
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for by the fact that post conversion, and with school capacity changes controlled for, Academy 
schools in the sample tended to admit pupils from a larger number of primary schools and from 
primary schools that performed better on average at KS2 than did either their predecessors or 
other non- Academy schools. This shows that the Academy effect partially reflects changes to 
intake sampling among Academy schools. The policy effect estimate drops from 2.409 to 1.573 
KS2 total points scores once consideration is made for the influence of these extra controls (see 
column 4 in Table 9 and column 7 in Table 13 respectively). With the characteristics of primary 
schools added, the estimated S coefficient is marginally reduced again, to 1.567 KS2 total points 
scores 67 . While these changes in the measured Academy effect are important, the d coefficient is 
still positive and maintains its statistical significance even after all of these factors have been 
taken into account. This is a result that has substantial implications. In particular, the sustained 
finding of an increase in intake quality among Academy schools once average primary school 
performance has been conditioned out suggests that these schools not only admitted more 
academically able pupils once they switched status, but some of these pupils attained KS2 
standards of achievement that were above the average for their primary school. As has been the 
case for all previous regression analyses, this outcome remains even after controlling for potential 
pupil capacity increases in Academy schools. Therefore, this provides further evidence of a 
changing intake ability profile in the sample of Academies that appears to reflect more pupil 
entry by higher ability pupils, including those with above (their primary- school) average test 
performance, at the expense of changes to pupil intake at other points in the attainment 
distribution. 

The crucial question that has yet to be answered is where along the ability distribution intake 
changes into Academy schools have taken place, with these adjustments having then enabled 
their intake to include more pupils of a relatively stronger average prior ability. In the final 
column of Table 13 results from an attempt to evaluate this issue are presented. Here estimation 
considers how the annual dispersion of KS2 intake ability into Academy schools compares with 
that in predecessor and control schools. In other words regression analysis assesses whether the 
year-on-year KS2 attainment range of pupils entering Academy schools looks narrower or wider 
than it was for their pre- Academy counterparts and for non-Academies. Given that the sample of 
Academies raised their admissions of pupils with higher prior attainment without this effect being 
fully absorbed by school size growth, then mean intake quality could have been pushed up in one 

67 Adding observable primary school characteristics to the regression leads to only a slight change in the coefficient 
because it is likely that the annual average KS2 performance of the primary school captures much the same 
information as is contained in the school-level attributes, since attainment is influenced by school-level contextual 
factors. 
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of two ways. Either Academy schools may have shifted towards an intake of pupils of a wider 
ability range once they switched status. In this case they might have raised their mean intake 
ability by sampling different fractions of pupils along the ability distribution, with a likely 
increase in the percentages admitted from the mid-points and above. Otherwise, following 
conversion, Academies may instead have lowered the spread of their pupil intake ability, cutting 
the proportion of pupil intake from the bottom end of the attainment distribution in particular, 
resulting in a rise in average intake ability into the school. In the second case, raising average 
pupil entry quality through reducing the intake ability spread will always be associated with a cut 
in the proportion of lower attaining pupils entering the school, while at the higher end of the 
performance distribution different scenarios could have taken shape. More specifically, one of 
three situations might have occurred that would allow for a reduced dispersion and higher mean 
ability among pupil admissions in post-conversion Academy schools. These are: (i) Academies 
could have cut only the proportion of lower-attaining pupils admitted to the school, leaving the 
intake composition along all other parts of the ability distribution unchanged; (ii) Academies 
could have reduced intake ability proportions at both ends of the performance spread, but they 
may have cut off relatively more pupils from the bottom than the top end; or (iii) Academies 
could have lowered admission shares at the bottom end and raised the pupil entry proportion at 
the upper end of the attainment distribution, but with an increase at the top end being of relatively 
smaller magnitude than the cut at the bottom end. In all cases, a rise in mean intake quality and a 
reduction in intake ability dispersion is the end result, an outcome brought about by slicing the 
entry share of pupils into the Academy school that are of an academically weaker background. 

In practice, dispersion changes can be assessed by re-estimating equation (2) using the annual 
standard deviation in KS2 total points scores as the outcome measure, rather than the annual 
average of this variable. The results derived from this process are given in column 9 of the Table. 
It is interesting to find that once schools converted into Academies they reduced their intake 
ability dispersion: the S coefficient estimated on the effective years of Academy school status is 
measured as -0.514 standard deviation units and is statistically significant at the 10 per cent 
significance level. Thus it would appear that there are proportionally fewer pupils with poor prior 
attainment in the Academy schools than in their predecessors, a situation that will have been 
reached by one of the three means set out above. Determination of the exact way in which this 
change in intake ability dispersion has happened is beyond the scope of the current analysis, but 
forms an interesting area for future research exploration. 
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This important result goes some way towards answering the key question behind this research, 
namely whether Academy schools created under the Labour government evaluate towards being 
more inclusive or more ‘exclusive’ than their pre-treatment counterparts. The evidence shown 
here suggests that the attainment profile of pupils entering Academy schools reflects a more 
‘exclusive’ intake, in which there has been a reduction in the admission of pupils with a weaker 
KS2 performance record in Academies relative to in their pre- Academy versions. Empirical 
assessment has also revealed that existing Academy schools are catering for pupils of above 
average ability in the primary school from which they came and there is some indication that 
innate ability has featured more among admitted pupils than has learning progression. Thus it 
seems that school conversion into an Academy has been characterised by stratification in intake 
along the lines of the ability distribution relative to the prior situation. However, the raised 
academic quality of pupil admissions into Academy schools represents just one dimension of 
their changing pupil profile. In the section that follows further categories of composition are 
assessed in order to gain a fuller picture of the impact that this particular policy of institutional 
reform has had on the constitution of schools to which it was applied in contrast to other 
‘untreated’ schools. 


5 Empirical Results: Further Findings 


Assessing ther imensions of ntake uality and xamining hanges in hole- chool 
omposition 

Panel A of Table 14 presents findings from difference-in-differences regression estimation in 
which various attributes of pupils entering year - 7 of Academy schools are compared with those 
among pupils joining predecessor and non- Academy schools. Information on aspects of pupil 
background pertaining to new secondary school joiners is contained within PLASC, a data source 
that has been collected annually since January 2002 with a total of 6 waves available at the time 
of empirical analysis (August 2008), covering 2002 to 2007. Panel B of the Table looks at whole 
school compositional changes in the effective policy years and not just variations at the year 7 
entry level. These details are given in the school-level files (as described in section 3, ‘Data 
Description’), which stretch further back than PLASC and here 11 years of data on secondary 
schools comprising the period 1997 to 2007 are used. The shorter time-frame of PLASC 
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availability reduces the window of pre-policy observations that can be utilised to determine 
changes in the intake composition of Academies by 5 years (1997 to 2001). Therefore for the first 
cohort of Academies only one year of pre-policy data on the characteristics of pupil entrants 
exists. Throughout the entire analysis equation (2) is estimated on a different dependent variable 
as shown in the column headings to Table 14. 

Column 1 begins by looking at changes in the fraction of pupil intake that is eligible for Free 
School Meals (FSM) in schools that turned into Academies. This indicator is frequently used as a 
proxy for social disadvantage, given that eligibility is means-tested and depends on family 
earnings falling below a certain minimum income threshold (see Appendix 1 for further details). 
As can be seen from the findings of this regression, intake into Academy schools has consisted of 
a lower proportion of FSM eligible pupils than was previously the case. In the common pre- 
policy year of 2002 the average percentage of FSM eligible pupils in year 7 of predecessor 
schools was 44.17 per cent. In the Academy years of these schools, the mean fell by 5.563 
percentage points to take the benchmark average to 38.61 per cent, a drop of 12.59 per cent. This 
suggests that the intake composition of Academy schools moved away from consisting of pupils 
from relatively deprived backgrounds to quite an extent, a finding that is statistically significant 
and occurs even though school size changes in Academies have been controlled for. 

In columns 2 to 6 of Table 14 (Panel A) consideration is made for whether other aspects of pupil 
characteristics differ in the Academy school years for pupils who started their secondary phase of 
education in the sample window. Columns 2 and 3 look for changes in the percentages of pupils 
with Special Educational Needs in Academies, either with or without a statement respectively. 
Column 4 assesses variations in the ethnic mix of pupil intake and in column 5 changes in the 
proportion of pupils with English as an additional language in Academy schools are evaluated. 
Column 6 looks at how the gender balance of Academies compares with that in predecessor and 
control group schools. The results of regression estimation reveal that none of these dimensions 
of intake composition changed to a discemable or statistically valuable degree in the years since 
the policy period has been in place. The same general finding stems from the analysis of changes 
in whole school features. Panel B of Table 14 shows that the percentages of pupils eligible for 
free school meals, those with SEN of any status and those classified as white ethnic origin in 
predecessor schools were left unaltered by the application of the Academies programme to these 
schools. 
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At this stage, empirical evaluation has highlighted that, apart from prior attainment, the only 
characteristic of pupil intake that has changed in a significant and substantial way in the 
Academy school years is FSM eligibility. Columns 7 and 8 in Panel A of Table 14 gauge whether 
there is any relation between these two intake categories that can enhance knowledge of the 
policy outcome, and the direction in which any association flows. Column 7 repeats the 
estimation procedure of column 1 and adds to this a control for the school-level average annual 
KS2 attainment of year 7 pupils, or in other words, KS2 intake quality. What this shows is that 
the large negative and statistically significant change in the percentage of pupils eligible for FSM 
in year 7 of Academy schools in comparison to predecessor schools that was found in column 1 
remains. The coefficient (standard error) on KS2 intake quality (not shown in the Table) stands at 
-0.572 (0.089) and is of high statistical content. The way to interpret this result is that a FSM 
eligible pupil with equivalent prior attainment to another pupil who is not eligible for FSM is 
statistically significantly less likely to have entered a school that converted into an Academy. 
Column 8 goes back to the estimated regression shown in column 4 of Table 9 and includes as 
another explanatory variable the percentage of year 7 pupils who are eligible for FSM. The 
addition of this further regressor does little to change the estimated S coefficient, which remains 
positive and statistically significant, at 2.049 KS2 total point scores. The percentage of pupils in 
year 7 who are eligible for FSM has high predictive power, with a coefficient (standard error) of - 
0.061 (0.011) (not reported in the Table). The interpretation of this outcome is that if two FSM 
eligible pupils differ by their KS2 attainment, the pupil who achieved a higher mean result in 
these tests is statistically significantly more likely to have entered a school that switched to 
Academy status. Overall, analysis into further intake composition changes in Academies relative 
to predecessor and non-Academy schools has revealed that these schools have not only shifted 
their intake towards academically stronger pupils and reduced their admission of weaker- 
attaining students, but they have also tended to feature fewer pupils from deprived backgrounds. 
These findings are in direct contrast to a principal stated objective of the scheme established from 
its outset: that of Academy schools having a more inclusive and mixed ability pupil profile (see 
section 2, ‘Aims and Objectives’). 
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6 Summary and Discussion 


Over the lifetime of the former Labour government, education policy in the UK increasingly 
sought to raise school standards and the performance of individual pupils through the 
introduction of school renewal programmes that have targeted institutions at the lower end of the 
attainment distribution. Under these schemes schools deemed to be failing in their delivery of 
education have experienced a complete overhaul in their operations in order to generate their 
revival and subsequent return to the education market place as viable competitors. A reform 
strategy that has been progressively applied to state secondary schools since the early 2000s is 
that of the Academies Programme, where underachieving schools have been granted autonomy 
from LEA control and are guided towards better functioning by an external sponsor. The first 
wave of Academies opened from September 2002 and by the change of government (May 2010) 
there were 203 Academy schools in existence. Labour’s plans to extend the scheme to 15 per cent 
coverage of the secondary education phase by 2015 would have made this the most prominent 
form of school reconstitution in the education arena. 

School improvement in the shape of the Academies Programme started out in deprived inner city 
areas, aiming to tackle the legacy of access to poor quality schooling among underprivileged 
pupils and the subsequent inequalities in educational opportunities. The broadening geographical 
coverage of Academy schools during Labour’s time in power reflects an understanding of the 
lack of confinement of this scenario to urban areas. Though they became more widespread, there 
is a distinct shortfall in knowledge on the effectiveness of these renewed schools in turning 
around the circumstances of the pupils for whom they have meant to cater in the areas in which 
they have been set up. Evaluation carried out in this paper has sought to examine the issue of the 
inclusiveness of Labour’s Academies scheme using information on schools that underwent 
conversion to the renewed status between 2003 and 2007. The analytical stance taken here has 
focused on compositional changes in these schools relative to their predecessors and to other 
schools within the LEA of Academies who have shared a similar historical evolution in their 
characteristic make-up but have differed by their non-participation in the strategy of school 
reform. Two different angles of composition have been investigated, these being relative changes 
in both the intake composition and the whole school pupil profile of Academy schools. 
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The results of empirical difference-in-differences evaluation undertaken here offer up interesting 
findings with regard to the initial period of this particular programme of school reform. The 
dimensions of intake that appear to have changed the most in Academy schools are the prior 
attainment distribution of pupils joining these schools and the percentage of new entrants with 
eligibility for free school meals. There has been a distinct and robustly estimated rise in intake 
ability among Academies as soon as they have re-opened under their revived status and a 
significant drop in the number of new pupils from deprived social backgrounds, patterns of 
change that did not occur in predecessor schools and that have not shown up in non-Academy 
control schools. Growth in the pupil capacity of these schools does not explain away these 
measured effects, implying that composition changes have been achieved through re-drawing the 
fractions of pupils admitted to the school from within the ability and social background ranges. 
Evidence presented here suggests that Academy schools have raised the average quality of their 
intake by lowering their admissions of weaker attaining students. Accordingly, school renewal of 
this kind appears to have resulted in a more ‘exclusive’ pupil profile within Academies and 
reduced entry into these schools of pupils that may have otherwise lowered the general academic 
performance of the school. In this respect education inequalities and schooling stratification along 
the lines of ability and social background have increased as a result of the compositional changes 
that Academy schools have made. This suggests that the “ultimate” objective of raising levels of 
achievement in the school (aim (1)) has taken precedence, to the detriment of aim (3) of the 
scheme, which seeks to raise the life chances of cohorts of deprived pupils through inclusive 
access to the renewed school (see section 2, ‘Aims and Objectives’). 

Further analysis of the outcomes of the original Academies programme beyond the years 
analysed here (1997 to 2007) - to incorporate all data on the scheme up to the point where the 
policy change of the Coalition government came into effect, that is, up to the academic year 
2009/2010 - would be beneficial for determining the extent to which these findings are a 
consequence of a small sample size. If the popularity of Academy schools gained further 
momentum after 2007, physical capacity constraints may have prevented the ability of these 
schools to admit an ever growing number of pupils. This may have furthered the degree and types 
of compositional change occurring in these schools over time. The difficult issue to empirically 
pinpoint from the analysis undertaken so far is whether the driver of exclusivity through 
compositional change has been the school or parents. Academies are their own admissions 
authority and therefore control the allocation of admissions, while in LEA-governed schools 
pupil entry is decided by the LEA. This characteristic also allows Academies to set their own 
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admissions rules, including those to be used in the event of oversubscription, so long as all rules 
applied comply with the mandatory requirements of the School Admissions Code. Research has 
suggested the need to establish whether relative admissions autonomy is responsible for 
compositional change. As West et al. (2009, pp. 5) note, “[k]ey questions remain in relation to 
the link between admissions criteria and practices and school composition.. ..it is still unclear 
whether school autonomy in relation to school admissions may be a factor in determining which 
pupils apply to which schools and which are offered places.” Meanwhile, Tough and Brooks 
(2007) cite research by the Sutton Trust which implicates admissions autonomy as the instrument 
for compositional change. The authors write that “[s]chools within the top 200 comprehensives 
that are their own admission authorities are also highly unrepresentative of the postcode sector in 
which they are located. Within these schools just 5.8 per cent of pupils are eligible for free school 
meals compared to 13.7 per cent of the pupils in their local area. By contrast, the other schools in 
the top 200 whose admissions are run by the local authority are roughly representative of their 
area” (ibid., pp. 16). To the extent that the ability of a school to set rules of entry changes the 
types of parents who apply to the school, compositional change reflects the interdependence of 
parental and school selection processes. 

As an initial detailed study into the effectiveness of the Academy model of school reconstitution 
in delivering its objective of raised inclusion, this work flags up concerns about the benefits of 
the original policy and also suggests the need for a thorough evaluation of the consequences of 
the scheme going forward. The Coalition party has shifted the focus of the Academies 
programme away from underperforming secondary schools and towards all institutions, with a 
particular emphasis on status switch among those schools with an outstanding record of academic 
achievement in the first instance. The revised policy offers some specific help for failing schools, 
insofar as outstanding schools that become Academies are required to mentor struggling schools 
within their area and steer them in the direction of improved standards. Additionally, to help 
underachieving and disadvantaged pupils, the government is pushing forward a more effective 
and targeted system of pupil-led funding through their ‘pupil premium’ policy, in which the extra 
monies that schools receive for admitting disadvantaged students will be made to more closely 
follow each pupil to whichever school they attend. This should enable schools that teach these 
students to directly acquire extra funds according to the numbers they admit, and to use these 
funds to raise entrenched low levels of attainment among this group. However, it remains to be 
seen whether these policy scenarios will generate any specific or indeed general improvements in 
circumstances for underprivileged students who frequently comprise the majority group at the 
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Table 1: School-level Difference-in-Differences Estimates of Academy Status on GCSE Performance, 1995/96 to 2005/06 


Panel A: Comparison with matched schools 


Academies opening in 
September 2002 
(1) (2) 

Academies opening in 
September 2003 
(3) (4) 

Academies opening in 
September 2004 
(5) (6) 

Academies opening in 
September 2005 
(V) (8) 

Becomes Academy 

-0.49 

1.57 

3.80 

0.01 

10.00** 

8.95* 

1.86 

-0.15 


(3.56) 

(3.88) 

(3.11) 

(3.42) 

(4.18) 

(5.01) 

(3.73) 

(3.85) 

School fixed effects 

Yes (6) 

Yes (6) 

Yes (18) 

Yes (18) 

Yes (6) 

Yes (6) 

Yes (19) 

Yes (19) 

Year dummies 

Yes (9) 

Yes (9) 

Yes (9) 

Yes (9) 

Yes (9) 

Yes (9) 

Yes (9) 

Yes (9) 

Tine-varying controls 

No 

Yes 

No 

Yes 

No 

Yes 

No 

Yes 

R-squared 

0.80 

0.85 

0.68 

0.81 

0.64 

0.66 

0.95 

0.94 

Number of schools 

7 

7 

21 

17 

6 

6 

19 

17 

Panel B: Comparison with all other state schools in LEA 


Academies 

opening in 

Academies 

opening in 

Academies 

opening in 

Academies 

opening in 


September 2002 

September 2003 

September 2004 

September 2005 


(9) 

(10) 

(ID 

(12) 

(13) 

(14) 

(15) 

(16) 

Becomes Academy 

3.02 

3.58 

8.45** 

4.12 

8.27** 

4.58 

3.01 

2.86 


(3.01) 

(3.44) 

(2.52) 

(2.71) 

(2.67) 

(3.36) 

(2.24) 

(2.14) 

School fixed effects 

Yes (27) 

Yes (27) 

Yes (116) 

Yes (116) 

Yes (89) 

Yes (89) 

Yes (201) 

Yes (201) 

Year dummies 

Yes (9) 

Yes (9) 

Yes (9) 

Yes (9) 

Yes (9) 

Yes (9) 

Yes (9) 

Yes (9) 

Tine-varying controls 

No 

Yes 

No 

Yes 

No 

Yes 

No 

Yes 

R-squared 

0.84 

0.85 

0.84 

0.87 

0.87 

0.89 

0.87 

0.88 

Number of schools 

26 

26 

110 

94 

81 

81 

186 

120 


Source: Adapted from Machin and Wilson (2008). 


Notes: Coefficient estimates pertain to difference-in-differences regression equations in which the dependent variable is the school-level percentage of pupils getting 5 
or more GCSEs graded A*-C. Robust standard errors are shown in parentheses. Control variables are time-varying school characteristics as follows: log(school size), 
proportion of pupils eligible for free school meals, proportion of pupils of non-white ethnic origin. School fixed effects control for characteristics of schools that are 
unchanging over time. ** indicates statistical significance at the 5% significance level, or better; * indicates significance at the 10% level. 
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Table 2: Academy Schools Opening Between 2002 and 2006 


Academy 

(1) 

Date of 
opening 

(2) 

Local Authority 
District (LAD) 
and region 

(3) 

LAD 

deprivation 

ranking 

(4) 

Sponsor(s) and contribution 
pledged (£ million) 

(5) 

Subject 

specialism(s) 

(6) 

Other details 

(7) 

Bexley Business 
Academy 

2002: 

September 1 st 

Bexley; Outer 
London 

194 

Sir David Garrard (2.41) 

Business & 
Enterprise 

- 

Greig City 
Academy 

2002: 

November 6 th 

Haringey; Outer 
London 

18 

Greig Trust and the Church of 
England (2.0) 

Technology 
(especially (ICT) 

- 

Unity City 
Academy 

2002: July 

31 st 

Middlesbrough; 
North East 

9 

Amey pic (2.0) 

Applied Enterprise 

Replaced 2 predecessor 
schools 

Capital City 
Academy 

2003: June 
12 th 

Brent; Outer 
London 

53 

Sir Frank Lowe (2.0) 

Sports and The 
Arts 

New building 

City of London 
Academy 

2003: June 2 nd 

Southwark; Inner 
London 

26 

Corporation of London (2.0) 

Business & 
Enterprise and 
Sports 

New school 

Djanogly City 
Academy 

2003: July 2 nd 

Nottingham; East 
Midlands 

13 

Sir Harry Djanogly (contributed £2 
mn to the school when it was a CTC 
- no extra contribution made in 
Academy conversion) 

ICT 

Replaced 2 predecessor 
schools, including 
Djanogly CTC 

The King's 
Academy 

2003: June 3 rd 

Middlesbrough; 
North East 

9 

Emmanuel Schools Foundation 
(2.0) 

Business & 
Enterprise 

Replaced 2 predecessor 
schools; 2 nd Academy to 
open in LEA 

Manchester 

Academy 

2003: 

September 1 st 

Manchester; 
North West 

4 

United Learning Trust and 
Manchester Science Park Ltd (2.0) 

Business & 
Enterprise and Art 

- 

The City 
Academy 

2003: June 3 rd 

Bristol; South 
West 

64 

John Laycock and the University of 
the West of England (2.499) 

Sports 

- 

The West London 
Academy 

2003: May 2 nd 

Ealing; Outer 
London 

84 

Alec Reed (2.0) 

Sports and 
Enterprise 

New building 

The Academy at 
Peckham 

2003: June 3 rd 

Southwark; Inner 
London 

26 

Harris Federation of South London 
Schools Trust (2.0) 

Business and 
Enterprise and the 
Performing Arts 

New building; 2 nd 
Academy to open in LEA 
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Table 2 (continued): Academy Schools Opening Between 2002 and 2006 


Academy 

(1) 

Date of 
opening 

(2) 

Local Authority 
District (LAD) 
and region 

(3) 

LAD 

deprivation 

ranking 

(4) 

Sponsor(s) and contribution 
pledged (£ million) 

(5) 

Subject 

specialism(s) 

(6) 

Other details 

(7) 

Walsall City 
Academy 

2003: 

September 1 st 

Walsall; West 
Midlands 

45 

Thomas Telford Online and the 
Mercers' Company (2.5) 

Technology 

New building 

Lambeth 

Academy 

2004: 

September 1 st 

Lambeth; Inner 
London 

19 

United Learning Trust (2.0) 

Business & 
Enterprise and 
Languages 

New school 

London Academy 

2004: 

September 1 st 

Barnet; Outer 
London 

128 

Peter Shalson (1.5) 

Business & 
Enterprise and 
Technology 

- 

Mossbourne 

Community 

Academy 

2004: 

September 1 st 

Hackney; Inner 
London 

2 

Sir Clive Bourne (2.15) 

Technology 

New school 

Northampton 

Academy 

2004: 

September 1 st 

Northampton; 
East Midlands 

129 

United Learning Trust (2.0) 

Sports, Business & 
Enterprise 

New building 

Stockley 

Academy 

2004: 

September 1 st 

Hillingdon; Outer 
London 

157 

Barry Townsley and others (2.0) 

Science and 
Technology 

- 

Dixons City 
Academy 

2005: 

September 1 st 

Bradford; 
Yorkshire and 
The Humber 

32 

Dixons Academy Trust (0.651) 

Performing Arts 
and Product Design 

Replaced Dixons CTC 

Haberdashers’ 
Aske Hatcham 
College 

2005: 

September 1 st 

Lewisham; Inner 
London 

39 

Haberdashers Livery Company 
(0.705) 

ICT and Music 

Replaced Haberdashers' 
Aske's CTC 

Haberdashers’ 
Aske Knights 
Academy 

2005: 

September 1 st 

Lewisham; Inner 
London 

39 

Haberdashers Livery Company 
(0.296) 

ICT and Sports & 
Science 

2 nd Academy to open in 
LEA 

Harefield 

Academy 

2005: 

September 2 nd 

Hillingdon; Outer 
London 

157 

David Meller/Haig 
Oundjian/Jonathon Green (1.5) 

Sports 

2 nd Academy to open in 
LEA 
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Table 2 (continued): Academy Schools Opening Between 2002 and 2006 


Academy 

(1) 

Date of 
opening 

(2) 

Local Authority 
District (LAD) 
and region 

(3) 

LAD 

deprivation 

ranking 

(4) 

Sponsor(s) and contribution 
pledged (£ million) 

(5) 

Subject 

specialism(s) 

(6) 

Other details 

(7) 

MacMillan 

Academy 

2005: 

September 1 st 

Middlesbrough; 
North East 

9 

Macmillan Academy Trust (1.25) 

Science & PE & 
Outdoor Education 

Replaced MacMillan 
CTC; 3 ld Academy to 
open in LEA 

Marlowe 

Academy 

2005: 

September 1 st 

Kent; South East 

n/a 

Roger De Haan & Kent County 
Council (2.735) 

Business & 
Enterprise and the 
Performing Arts 

- 

Salford City 
Academy 

2005: 

September 1 st 

Salford; North 
West 

15 

United Learning Trust and 
Manchester Diocese (1.6) 

Business & 
Enterprise & Sports 

- 

St Paul’s 
Academy 

2005: 

September 1 st 

Greenwich; Inner 
London 

24 

Roman Catholic Archdiocese of 
Southwark (2.0) 

Sports and 
Enterprise 

- 

The Academy of 
St Francis of 
Assisi 

2005: 

September 1 st 

Liverpool; North 
West 

1 

Diocese of Liverpool/RC 
Archdiocese of Liverpool (2.0) 

The Environment 

New building 

Trinity Academy 

2005: 

September 1 st 

Doncaster; 
Yorkshire and 
The Humber 

41 

Emmanuel Schools Foundation 
(2.0) 

Business & 
Enterprise 

New building 

Barnsley 

Academy 

2006: 

September 1 st 

Barnsley; 
Yorkshire and 
The Humber 

43 

United Learning Trust (1.5) 

Science with 
Business & 
Enterprise 

- 

Burlington Danes 
Academy 

2006: 

September 1 st 

Hammersmith & 
Fulham; Inner 
London 

59 

Absolute Return for Kids (ARK) 
(1.5) 

Expressive Arts & 
Maths 

- 

David Young 

Community 

Academy 

2006: 

September 1 st 

Leeds; Yorkshire 
and The Humber 

85 

Diocese of Ripon and Leeds (1.5) 

Design and the 
Built Environment 

Replaced 2 predecessor 
schools 

Gateway 

Academy 

2006: 

September 1 st 

Thurrock; East of 
England 

124 

The Ormiston Trust (unknown) 

Arts and 
Engineering 

- 
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Table 2 (continued): Academy Schools Opening Between 2002 and 2006 


Academy 

(1) 

Date of 
opening 

(2) 

Local Authority 
District (LAD) 
and region 

(3) 

LAD 

deprivation 

ranking 

(4) 

Sponsor(s) and contribution 
pledged (£ million) 

(5) 

Subject 

specialism(s) 

(6) 

Other details 

(7) 

Grace Academy 

2006: 

September 1 st 

Solihull; West 
Midlands 

199 

Bob Edmiston (2.0) 

Business & 
Enterprise 

- 

Harris Academy 

2006: 

September 1 st 

Merton; Outer 
London 

222 

Harris Federation of South London 
Schools Trust (0.5) 

Sports & Enterprise 

- 

Harris Girls 
Academy 

2006: 

September 1 st 

East Dulwich; 
Inner London 

n/a 

Harris Federation of South London 
Schools Trust (0.5) 

Sports & PE Health 

- 

Landau Forte 
College 

2006: 

November 

30 th 

Derby; East 
Midlands 

69 

Landau Charitable Foundation and 
Rocco Forte Hotels Pic (0.46) 

Technology & 
Business Enterprise 

Replaced Landau Forte 
CTC 

North Liverpool 
Academy 

2006: 

September 1 st 

Liverpool; North 
West 

1 

Liverpool University & Granada 
Learning (1.0) 

Business & 
Enterprise 

Replaced 2 predecessor 
schools; 2 nd Academy to 
open in LEA 

Paddington 

Academy 

2006: 

September 1 st 

Westminster; 
Inner London 

72 

United Learning Trust (1.5) 

Media & 
Performing Arts 
with Business & 
Enterprise 

Replaces same 
predecessor school as 
Westminster Academy 

Sandwell 

Academy 

2006: 

September 1 st 

Sandwell; West 
Midlands 

14 

Mercers Company, Thomas Telford 
Online, HSBC, West Bromwich 
Football Club (2.794) 

Business & 
Enterprise & Sports 

New school 

Sheffield Springs 

2006: 

September 1 st 

Sheffield; 
Yorkshire and 
The Humber 

63 

United Learning Trust (1.5) 

Performing Arts 
and Technology 

- 

Sheffield Park 

2006: 

September 1 st 

Sheffield; 
Yorkshire and 
The Humber 

63 

United Learning Trust (1.0) 

Business & 
Enterprise 

2 nd Academy to open in 
LEA 

St Mark's Church 
of England 
School 

2006: 

September 1 st 

Merton; Outer 
London 

222 

Southwark Diocese/CfBT 
Education Trust/Toc H charity 
(unknown) 

Science, Enterprise 
& Technology 

2 nd Academy to open in 
LEA 
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Table 2 (continued): Academy Schools Opening Between 2002 and 2006 


Academy 

(1) 

Date of 
opening 

(2) 

Local Authority 
District (LAD) 
and region 

(3) 

LAD 

deprivation 

ranking 

(4) 

Sponsor(s) and contribution 
pledged (£ million) 

(5) 

Subject 

specialism(s) 

(6) 

Other details 

(7) 

The John 
Madejski 
Academy 

2006: 

September 1 st 

Reading; South 
East 

151 

John Madejski (2.0) 

Sports 

- 

The Harris 

Bermondsey 

Academy 

2006: 

September 1 st 

Southwark; Inner 
London 

26 

Harris Federation of South London 
Schools Trust (1.5) 

Enteiprise & Media 

3 rd Academy to open in 
LEA 

The Petchey 
Academy 

2006: 

September 1 st 

Hackney; Inner 
London 

2 

Jack Petchey Foundation (2.0) 

Health Care and 
Medical Sciences 

New school; 2 nd 
Academy to open in LEA 

Walthamstow 

Academy 

2006: 

September 1 st 

Waltham Lorest; 
Outer London 

27 

United Learning Trust (1.5) 

Business & 
Enterprise and 
Science & Maths 

- 

Westminster 

Academy 

2006: 

September 7 th 

Westminster; 
Inner London 

72 

Exilarch's Foundation (2.0) 

International 
Business & 
Enterprise 

Replaces same 
predecessor school as 
Paddington Academy; 2 nd 
Academy to open in LEA 


Sources: Machin and Wilson (2008); DCSF Standards Site “Current Projects of the Academies Programme” (see spreadsheet on “Open Academies” as at January 2009: 
http://www.standards.dfes. gov. uk/gas test7ian09acadslist.xls) : and author’s own searches into individual Academy school websites. Column 3 uses DCSF-provided Edubase 
dataset on the Register of Educational Establishments (REE) in England in 2006/2007, and Local Government Finance Statistics England No. 18 (2008; Map Alf, pp. 134). 
Column 4 uses Indices of Deprivation 2007, LA Summaries ID 2007 (see http://www.communities.gov.uk/documents/communities/xls/576504.xls (accessed 1 March 2009)). 
Column 5 uses Hansard (2008c) and The TES (2006) for details on sponsor(s) pledged contributions. For columns 3 and 4, there are 354 LADs in England. For column 7: unless 
otherwise stated, each Academy replaces one predecessor school. 
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Table 3: Share of Academy Schools in All State Secondary Schools, 2006/07 



All state secondary 

Academies 

Academies share 

Number of schools 

3,178 

46 

0.014 

Number of FTE pupils 

3,110,347 

41,437 

0.013 

Number of FTE teachers 

188,794 

2,751 

0.015 

Pupil-teacher ratio 

16.47 

15.06 

n/a 


Sources: DCSF-provided Edubase dataset (on the Register of Educational Establishments (REE) in England) and Annual School Census (ASC) 
dataset, both for 2006/07. The abbreviation FTE stands for full-time equivalent. 


Table 4: Number of Pupils Entering Year 7 of the Secondary Schools Sample and Their Match to KS2 Prior Attainment 


Data source and 
academic year 

Key Stage 3 

PLASC 

1998- 

1999- 

2000- 

2001- 

2002- 

2001- 

2002- 

2003- 

2004- 

2005- 

2006- 

1999 

2000 

2001 

2002 

2003 

2002 

2003 

2004 

2005 

2006 

2007 

Year of entry into 
secondary school 
(year 7) 

1996- 

1997- 

1998- 

1999- 

2000- 

2001- 

2002- 

2003- 

2004- 

2005- 

2006- 

1997 

1998 

1999 

2000 

2001 

2002 

2003 

2004 

2005 

2006 

2007 

No. of pupils in 
year 7 

121,829 

123,397 

125,962 

129,134 

127,862 

128,316 

128,453 

128,057 

124,012 

121,867 

120,486 

No. of pupils in 
year 7 with linked 
KS2 attainment 

109,124 

114,220 

118,679 

121,511 

121,448 

124,829 

125,393 

125,004 

120,469 

118,593 

117,160 

Percentage linked 

89.57% 

92.56% 

94.22% 

94.10% 

94.98% 

97.28% 

97.62% 

97.62% 

97.14% 

97.31% 

97.24% 


Notes: KS3 exams are taken when pupils are aged 13/14, in year 9 of secondary school. Assuming no school mobility over the period, pupils who took their KS3 exams in a 
particular secondary school should have entered the same secondary school two academic years earlier, aged 11/12 (year 7). KS2 tests are taken one school year prior to year 7 
entry into secondary school, when pupils are in the last year of Primary school and are aged 10/11. 
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Table 5: Number of Academy and Non-Academy Secondary Schools and the Structure of Academy Cohorts 


Panel A: School sizes and number of LEAs in the original sample and final balanced panel sample (1997-2007) 


Category 

Original sample size 
Balanced panel sample size 

Academy schools 
46 
33 

Non-Academy schools 
1,699 
389 

LEAs 

34 

25 

Panel B: Structure and size of the Academy school cohorts; structure of the non-Academy group of schools (1997-2007) 

Year of entry into 
secondary school (year 7) 

1996- 

1997 

1997- 

1998 

1998- 

1999 

1999- 

2000 

2000- 

2001 

2001- 

2002 

2002- 

2003 

2003- 

2004 

2004- 

2005 

2005- 

2006 

2006- 

2007 

Academy Cohort 1 

P 

P 

P 

P 

P 

P 

A 

A 

A 

A 

A 

Original number of Academy schools in cohort 1 = 3; Balanced panel number of Academy schools in cohort 1 = 3 

Academy Cohort 2 

P 

P 

P 

P 

P 

P 

P 

A 

A 

A 

A 

Original number of Academy schools in cohort 2 = 9; Balanced panel number of Academy schools in cohort 2 = 6 

Academy Cohort 3 

P 

P 

P 

P 

P 

P 

P 

P 

A 

A 

A 

Original number of Academy schools in cohort 3 = 5; Balanced panel number of Academy schools in cohort 3 = 2 

Academy Cohort 4 

P 

P 

P 

P 

P 

P 

P 

P 

P 

A 

A 

Original number of Academy schools in cohort 4 = 10; Balanced panel number of Academy scl 

tools in col 

tiort 4=10 

Academy Cohort 5 

P 

P 

P 

P 

P 

P 

P 

P 

P 

P 

A 

Original number of Academy schools in cohort 5 = 19; Balanced panel number of Academy scl 

tools in col 

tiort 5 = 12 

All other schools in 
Academy LEAs 

U 

U 

U 

U 

u 

U 

U 

U 

U 

U 

U 


Notes: ‘P’ in the Table indicates the predecessor school years (prior to the switch to Academy school status) for each cohort. ‘A’ indicates the initial year in which the schools in 
each cohort became Academies and all subsequent years of Academy school status thereafter. ‘U’ stands for unchanged, to represent all other state secondary schools located in an 
LEA containing at least one Academy, where these other schools did not become Academies themselves. 
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Table 6: Descriptive Statistics of School-Level Characteristics 


Panel A: School-level characteristics of predecessor and non-Academy secondary schools, 
1997-2002 averages 



(1) 

(2) 

(3) 

(4) 


Predecessor 

Non- Academy 

Difference 

T-statistic of 

Variable 

schools 

schools 

(l)-(2) 

difference 

% eligible for Free 

41.31 

25.18 

16.13 

5.84* 

School Meals 

(15.81) 

(15.19) 



% with SEN, 

3.21 

3.89 

-0.68 

-0.85 

with statement 

(1.78) 

(4.59) 



% with SEN, 

24.40 

19.57 

4.83 

3.23* 

no statement 

(9.00) 

(8.19) 



% white 

69.18 

77.53 

-8.35 

-1.73 


(27.19) 

(26.51) 



School size 

910 

1020 

-110 

-1.93 

(number of pupils) 

(345) 

(312) 



Pupil-teacher ratio 

15.13 

15.60 

-0.47 

-1.93 


(1.59) 

(1.32) 



% 5+ GCSEs, A*-C 

25.45 

38.34 

-12.89 

-4.33* 


(19.61) 

(16.11) 



% no passes at 

22.25 

12.46 

9.79 

6.55* 

GCSE 

(11.98) 

(7.85) 



Number of secondary 

33 

389 



schools 





Panel B: Characteristics of Primary schools attended by pupils entering into predecessor 

and non-Academy schools, 1997-2002 averages 



(1) 

(2) 

(3) 

(4) 

\ / O V A O r\ 1 

Predecessor 

Non- Academy 

Difference 

T-statistic of 

v ariaoie 

schools 

schools 

(l)-(2) 

difference 

% eligible for Free 

39.14 

26.83 

12.31 

5.60* 

School Meals 

(10.46) 

(12.25) 



% with SEN, 

3.40 

2.50 

0.90 

3.07* 

with statement 

(1.96) 

(1.58) 



% with SEN, 

22.76 

20.35 

2.41 

2.35* 

no statement 

(5.58) 

(5.69) 



% white 

70.04 

78.10 

-8.06 

-1.87 


(23.69) 

(23.78) 



School size 

398 

343 

55 

4.16* 

(number of pupils) 

(153) 

(62) 



Pupil-teacher ratio 

21.34 

21.66 

-0.32 

-0.80 


(2.03) 

(2.17) 



Average KS2 

71.00 

74.56 

-3.56 

-4.89* 

performance 





(points score) 

(2.95) 

(4.10) 



Mean number of 

36 

34 



primary schools 






Note: The standard deviation of each variable is shown in parentheses. * indicates statistical significance at the 5% 
level, or better. SEN stands for Special Educational Needs. 
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Table 7: Models of Academy School Probability: Pr(Academy) = 1, logit marginals and 
percentage effects, 1997-2002 averaged characteristics 


Variable 

Model 1 
(1) 

Marginal 

effects 

Full Controls 
(2) 

% effect on 
Pr(Academy)=l 

Model 2 
(3) 

Marginal 

effects 

Selected Controls 
(4) 

% effect on 
Pr(Academy)=l 

% eligible for Free 
School Meals 

0.0016 

(0.0011) 

[0.0013] 

5.39 

0.0013 

(0.0009) 

[0.0012] 

3.90 

% with SEN, with 
statement 

-0.0037 

(0.0027) 

[0.0027] 

-12.94 



% with SEN, 
no statement 

0.0004 

(0.0013) 

[0.0015] 

1.47 



% white 

0.0005 

(0.0005) 

[0.0005] 

1.90 

0.0005 

(0.0005) 

[0.0005] 

1.67 

School size 
(number of pupils) 

0.0000 

(0.0000) 

[0.0000] 

0.00 

0.0000 

(0.0000) 

[0.0000] 

0.01 

Pupil-teacher ratio 

-0.0077 

(0.0068) 

[0.0073] 

-26.82 

-0.0080 

(0.0084) 

[0.0086] 

-24.75 

% 5+ GCSEs, A*-C 

0.0008 

(0.0012) 

[0.0013] 

2.79 



% no passes at GCSE 

0.0034* 

(0.0017) 

[0.0017] 

11.64 

0.0031* 

(0.0014) 

[0.0014] 

9.42 

Pseudo R-Squared 

0.2692 

- 

0.2560 

- 

% correctly predicted, 
Academy schools 

98.35 

- 

97.80 

- 

% correctly predicted, 
Non-Academy schools 

92.97 

- 

93.11 

- 


Notes: The Table shows marginal effects from logit models based on whole school-level controls averaged over 
1997-2002; robust standard errors are shown in round parentheses, clustered standard errors (clustered at the LEA 
level) are shown in square brackets. Models are based on 422 schools, of which 33 are Academy schools and 389 are 
non-Academies. * indicates a statistically significant marginal effect at the 5% level of significance, or better. The 
dependent variable is a dichotomous indicator, taking the value of one if a school is an Academy and zero otherwise, 
where the dummy covers all five Academy cohorts (see Table 5 and Table 8 for the number of Academy schools in 
each cohort). The predicted probabilities of a school becoming an Academy are 2.88% and 3.24% for logit models 1 
and 2 respectively. This compares with 7.82% of schools that are Academies in the sample. Both specifications 
additionally include LEA dummies to control for time-invariant, LEA-specific factors that have the same impact on 
all schools within an LEA. 
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Table 8: Average Annual Key Stage 2 Total Points Scores of Year 7 Pupils by Academy/Non- Academy Schools within the CSR and by each 

Academy School Cohort (1997-2007) 



Notes: The Table shows the average annual Key Stage 2 total points scores of year 7 pupils in grouped Academy and non-Academy schools and by each Academy cohort over the 
period 1997-2007. The sample includes pupils in Academy and non-Academy schools belonging to the common support region determined by the logit regression as defined in 
Table 7, model 2 (see also Figure 2). Boxed italic figures indicate the policy on years for each Academy cohort. Coefficient estimates and associated robust standard errors 
(clustered at the school level and shown in parentheses) given in column 13 are based on a simple difference-in-differences regression equation of the outcome variable on an 
Academy school dummy, an Academy*policy on indicator, and year dummies, where the change in intake quality for Academy versus non-Academy schools uses the first (1997) 
and last (2007) years of average KS2 total points scores only. * indicates statistical significance at the 5 % level, or better. 
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Table 9: School-Level Difference in-Differences Estimates of the Effect of Academy Status 

on Key Stage 2 Intake (1997-2007) 



(1) 

(2) 

(3) 

(4) 

Academy on effect (all academies) 

2.460* 

2.460* 

2.460* 

2.409* 

(academy*policyon) 

(0.547) 

(0.549) 

(0.574) 

(0.575) 

Cohort 1 

-6.486* 

-7.168* 

- 

- 


(0.508) 

(1.766) 



Cohort 2 

-7.588* 

-7.863* 

- 

- 


(1.048) 

(0.820) 



Cohort 3 

-4.786* 

-8.745* 

- 

- 


(0.988) 

(1.183) 



Cohort 4 

0.222 

-0.141 

- 

- 


(2.471) 

(2.301) 



Cohort 5 

-4.123* 

-4.650* 

- 

- 


(1.507) 

(1.368) 



Year dummies 

Yes 

Yes 

Yes 

Yes 

LEA dummies 

No 

Yes 

No 

No 

School fixed effects 

No 

No 

Yes 

Yes 

School-level controls for school size 
and pupil-teacher ratio 

No 

No 

No 

Yes 


Note: The Table reports difference-in-differences regressions in which the dependent variable is the average annual 
KS2 total points score of year 7 pupils and explanatory variables for each specification are as listed. Robust standard 
errors (clustered at the school level) are shown in parentheses. All regressions use Academy and non-Academy 
schools belonging to the common support region determined by the logit regression as defined in Table 7, model 2 
(see also Figure 2), so that regressions are based on 3,949 observations covering 359 schools, of which 33 are 
Academies and 326 are non-Academies (see Table 8 for the number of Academy schools in each cohort). * indicates 
statistical significance at the 5% level, or better. The mean of the dependent variable in the common pre -policy year 
across all Academy cohorts (2002) is 73.577. 
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Table 10: Robustness checks 



(1) 

Excluding 
former CTCs 

(2) 

No logit 
common 
support 

(3) 

Logit common 
support, full 
controls 

(4) 

Probit common 
support, 
selected 
controls 

(5) 

School- 

specific 

trends 

(6) 

Years 

2002-2007 

(7) 

Fake policy 
on (years 
1997-2002) 

Academy on effect 
(all academies) 

3.046* 

2.547* 

2.400* 

2.358* 

2.136* 

2.388* 

0.148 

(academy*policyon) 

(0.538) 

(0.573) 

(0.577) 

(0.578) 

(0.794) 

(0.718) 

(0.475) 

Number of obse rvations 

3,553 

4,642 

3,289 

3,322 

3,949 

2,154 

2,154 

Number of schools 

323 

422 

299 

302 

359 

359 

359 

Of which academy schools 

29 

33 

33 

33 

33 

33 

33 

Of which iion-academy schools 

294 

389 

266 

269 

326 

326 

326 


Note: The Table shows difference-in-differences regressions in which the dependent variable is the average annual KS2 total points score of year 7 pupils. All regressions include 
controls as follows: year dummies, school fixed effects and school-level controls for school size and the pupil-teacher ratio. Robust standard errors (clustered at the school level) 
are shown in parentheses. * indicates statistical significance at the 5% level, or better. Regressions shown in columns (1), (5), (6), and (7) use Academy and non-Academy schools 
belonging to the common support region determined by the logit regression as defined in Table 7, model 2 (see also Figure 2). Column (1) excludes from this common support 
region former CTCs that became Academies and all associated non-Academy schools in their respective LEAs if a dropped CTC-turned Academy school represents the only 
Academy school in the LEA. The logit regression on which column (3) is based is model 1 of Table 7 (see also Figure 1). The probit regression on which column (4) is based uses 
the same selected controls as the logit regression shown in Table 7, model 2 The regression shown in column (5) is the same as that in Table 9, column (4) plus an additional 
control for school-specific trends, where the latter consists of an interaction term between each school dummy and a time counter for the year (1 to 11). 
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Table 11: The Structure of the Falsification Exercise as a Robustness Check (see columns 6 and 7 of Table 10) 



Notes: ‘P’ in the Table indicates the predecessor school years (prior to the switch to Academy school status) for each cohort. ‘A’ indicates the initial year in which the schools in 
each cohort became Academies and all subsequent years of Academy school status thereafter within the sample period. The actual predecessor years and Academy years for each 
Academy cohort shown here are the same as those in Table 5. The fake predecessor and Academy years for the cohorts are those corresponding to the experimental scenario 
where the Academy years are shifted back into the pre-policy and predecessor time period (see column 7 of Table 10). Outlined sections of the actual case indicate where the 
actual policy set up has been made to resemble the number of years and layout of the policy design of the fake experiment (see column 6 of Table 10). 
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Table 12: Testing for Dynamic Effects: Key Stage 2 Intake Changes during the Policy On 


Period in Academy Schools 



(1) 

Academy* 
policy on; 
time on 

(2) 

Academy 
*policy 
on by 
cohort 

(3) 

Academy 
*policy 
on by 
cohort; 
time on 

(4) 

Academy* 
policy on 
by cohort; 
time on by 
cohort 

Academy on effect (all academies) 
(academy*policyon) 

2.009* 

(0.832) 


“ 

- 

Time on effect (all academies) 
(academy*policyon*timeon) 

0.208 

(0.240) 


0.194 

(0.251) 


Academy on effect, cohort 1 

- 

2.351* 

1.770 

2.784 

(cohort 1 dummy *policy on) 


(0.582) 

(1.066) 

(1.571) 

Academy on effect, cohort 2 

- 

2.777* 

2.297 

1.356 

(cohort 2 dummy *policy on) 


(0.985) 

(1.227) 

(1.311) 

Academy on effect, cohort 3 

- 

3.701* 

3.319* 

3.181 

(cohort 3 dummy *policy on) 


(1.223) 

(1.349) 

(1.917) 

Academy on effect, cohort 4 

- 

1.596 

1.313 

1.697 

(cohort 4 dummy *policy on) 


(1.490) 

(1.511) 

(1.508) 

Academy on effect, cohort 5 

- 

2.627* 

2.443* 

2.639* 

(cohort 5 dummy *policy on) 


(0.852) 

(0.886) 

(0.853) 

Time on effect, cohort 1 

(cohort 1 dummy*policyon*timeon) 

- 



-0.145 

(0.421) 

Time on effect, cohort 2 

(cohort 2 dummy*policyon*timeon) 

- 



0.570* 

(0.255) 

Time on effect, cohort 3 

(cohort 3 dummy*policyon*timeon) 

“ 



0.264 

(0.354) 

Time on effect, cohort 4 

(cohort 4 dummy*policy-on*time-on) 

- 


“ 

-0.060 

(0.594) 

Testing “academy on ” effects by 
cohort jointly equal zero (p-value) 

- 

0.000 

0.022 

0.003 

Testing “academy on ” effects by 
cohort are jointly equal (p-value) 

- 

0.824 

0.780 

0.883 

Testing “time on ” effects by cohort 
jointly equal zero (p-value) 

- 

- 

- 

0.227 

Testing “time on ” effects by cohort 
are jointly equal (p-value) 

- 

- 

- 

0.438 


Note: The Table shows difference-in-differences regressions in which the dependent variable is the 
average annual KS2 total points score of year 7 pupils and explanatory variables consider different 
specifications of dynamic effects as listed, for years 1997-2007. All regressions include additional 
controls as follows: year dummies, school fixed effects and school-level controls for school size and 
the pupil-teacher ratio. Robust standard errors (clustered at the school level) are shown in 
parentheses. All regressions use Academy and non-Academy schools belonging to the common 
support region determined by the logit regression as defined in Table 7, model 2 (see also Figure 2), 
so that regressions are based on 3,949 observations covering 359 schools, of which 33 are 
Academies and 326 are non-Academies (see Table 8 for the number of Academy schools in each 
cohort). * indicates statistical significance at the 5% level, or better. 


88 


Table 13: Describing Mechanisms Behind KS2 Intake Changes in the Policy On Years 



(1) 

(2) 

(3) 

Dependent variable 

(4) (5) (6) 

(7) 

(8) 

(9) 


KS2 intake: 

using all 
KS2 pupils 
and 

restricted 

sample 

KS2 intake: 

using pupils 
with KS 1 
and KS2 
outcomes 

KS1 

levels 

KS1-KS2 

value- 

added 

No. of 
intake 
Primary 
schools 

Average 

KS2 

performanc 
e of intake 
Primary 
schools 

KS2 

intake: 

controlling 

for 

(5) and (6) 

KS2 intake: 

controlling for 
(5) and (6) 
and Primary 
school-level 
characteristics 

Dispersion 
of KS2 
intake 

Academy on effect 
(all academies) 

2.339* 

1.758* 

1.146 § 

0.611 

4.427* 

0.865* 

1.573* 

1.567* 

-0.514 § 

(academy*policyon) 
Mean of the dependent 

(0.838) 

(0.725) 

(0.673) 

(0.388) 

(1.585) 

(0.270) 

(0.500) 

(0.518) 

(0.283) 

variable for academies 
(2002 or 2003) 

73.69 

75.01 

39.14 

35.87 

33 

75.62 

73.58 

73.58 

16.05 


Note: School-level difference-in-differences regressions shown in columns (1) to (4) are based on a reduced sample of years and schools corresponding to pupils joining year 7 
between 2003 and 2007. These use 1,700 observations on 340 schools in total, of which 30 are Academies and 310 are non-Academies. Cohort 1 of Academy schools, opening 
from September 2002, has been excluded from these regressions due to a lack of pre -policy information on this cohort when pupils are required to have both KS1 and KS2 
outcomes. In cases where an Academy school within cohort 1 constitutes the only Academy school within the LEA, all schools in the LEA are dropped from the sample. 
Otherwise only the Academy school belonging to the initial cohort is dropped. Thus all schools in two LEAs and a total of three Academy schools are omitted from the sample of 
pupils in columns (1) to (4). Regressions shown in columns (5) to (9) are based on the years 1997-2007 and they use 3,949 observations covering 359 schools, of which 33 are 
Academies and 326 are non-Academies. In columns (1), (2), (7) and (8) the dependent variable is the average annual KS2 total points score of year 7 pupils; in column (3) it is the 
average annual KS1 total points score of year 7 pupils with KS2 outcomes; in column (4) it is the average annual KS1 to KS2 value-added of year 7 pupils; in column (5) it is the 
number of primary schools from which pupils in year 7 of secondary school came; in column (6) it is the average annual whole school KS2 performance of these primary schools, 
averaged at the secondary school level and in column (9) it is the average annual standard deviation in KS2 total points scores among year 7 pupils. For the regressions in columns 
(1) to (4) the mean of the dependent variable refers to 2003, otherwise it refers to 2002. All regressions include additional secondary school-level controls as follows: year 
dummies, school fixed effects and school-level controls for the school size and the pupil-teacher ratio. Column (8) includes additional primary school-level controls as follows: the 
fraction of FSM eligible pupils, the fraction of pupils with SEN with and without a statement, the fraction of pupils of white ethnic origin, school size, and the pupil teacher ratio. 
Robust standard errors (clustered at the school level) are shown in parentheses. All regressions use Academy and non-Academy schools belonging to the CSR determined by the 
logit regression as defined in Table 7, model 2 (see also Figure 2). * indicates statistical significance at the 5% level, or better. 8 indicates significance at the 10% level. 
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Table 14: Relation of Other Outcomes to Academy School Status: Pupils in Year 7 (2002-07) and All Pupils in the School (1997-07) 


PANEL A: Intake Composition 
(pupils in year 7; 2002-2007) 



(1) 

Eligible 
for Free 
School 
Meals 

(2) 

Special 
Educational 
Needs, with 
statement 

(3) 

Special 
Educational 
Needs, no 
statement 

(4) 

White 

(5) 

English as a 
second 
language 

(6) 

Gender 

(male=l) 

(7) 
FSM 
eligibility 
controlling 
for KS2 
outcomes 

(8) 

KS2 

outcomes 
controlling 
for FSM 
eligibility 

Academy on effect 
(all academies) 

-5.563* 

-0.417 

0.389 

0.302 

0.269 

0.680 

-4.197* 

2.049* 

(academy*policyon) 

(1.436) 

(0.326) 

(2.609) 

(1.125) 

(2.347) 

(1.005) 

(1.324) 

(0.693) 

Mean of the dependent variable 
for academies (2002 ) 

44.17 

3.70 

27.49 

69.47 

79.15 

49.96 

44.17 

73.58 


PANEL B: Whole School Composition 
(all pupils in the school; 1997-2007) 


Academy on effect 
(all academies) 

-1.697 

0.283 

1.090 

0.171 

- 

- 

- 

- 

(academy*policyon) 

(1.200) 

(0.396) 

(1.603) 

(4.718) 

- 

- 

- 

- 

Mean of the dependent variable 
for academies ( 1997-2002 ) 

41.31 

3.21 

24.40 

69.18 

- 

- 

- 

- 


Note: Panel A: Difference-in-differences regressions on changes to intake composition are based on the annual year 7 pupil-level data (available from PLASC for 2002-2007). 
Regressions use 2,154 observations, covering 359 schools, of which 33 are academies and 326 are non-academies. Panel B: Whole school difference-in-differences regressions 
are based on the years 1997-2007. Regressions use 3,949 observations, covering 359 schools as for Panel A. All regressions include additional controls as follows: year dummies, 
school fixed effects and school-level controls for the school size and the pupil-teacher ratio. Robust standard errors (clustered at the school level) are shown in parentheses. All 
regressions use Academy and non-Academy schools belonging to the common support region determined by the logit regression as defined in Table 7, model 2 (see also Figure 
2). * indicates statistical significance at the 5% level, or better. 
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Figure 1: Propensity Scores for Academy and Non- Academy Schools: Logit Model with 

Full Controls (see Table 7, Model 1) 



Propensity Score 


I 1 Untreated schools (non-academies) Treated schools (academies) 


Note: Diagram plots histograms of the implied probability of treatment for Academy and non-Academy schools, where the 
probability estimates are predicted using the full logit specification as shown in Table 7 (model 1; see also Table 8, column 
3). The common support region of (0.0115 0.8068) includes 33 Academy schools (out of 33) and 266 non-Academy 
schools (out of 389). 


Figure 2: Propensity Scores for Academy and Non-Academy Schools: Logit Model with 

Selected Controls (see Table 7, Model 2) 



Propensity Score 


Untreated schools (non-academies) Treated schools (academies) 


Note: Diagram plots histograms of the implied probability of treatment for Academy and non-Academy schools, where the 
probability estimates are predicted using the selected logit specification as shown in Table 7 (model 2). The common 
support region of (0.0056 0.7919) includes 33 Academy schools (out of 33) and 326 non-Academy schools (out of 389). 
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Figure 3: Propensity Scores for Academy and Non- Academy Schools: Probit Model with 

Selected Controls (see Table 10, column 4) 



Propensity Score 


Untreated schools (non-academies) Treated schools (academies) 


Note: Diagram plots histograms of the implied probability of treatment for Academy and non-Academy schools, where the 
probability estimates are predicted using the selected probit specification as discussed in Table 10, column 4 (for the list of 
selected controls used in the probit model see Table 7, model 2). The common support region of (0.00917 0.7243) includes 
33 Academy schools (out of 33) and 269 non-Academy schools (out of 389). 
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Appendix 1 


Conditions for Free School Meal (FSM) Eligibility 

“Children whose parents receive the following are entitled to free school meals: 

Income Support (IS); 

Income Based Jobseekers Allowance (IBJSA); 

Support under part VI of the Immigration and Asylum Act 1999; 

Child Tax Credit, provided they are not entitled to working Tax Credit and have an annual 
income, as assessed by HM Revenue and Customs, that (for 2007/2008) does not exceed 
£14,495; or 

The guaranteed element of State Pension Credit; and 

Children who receive IS or IBJSA in their own right are also entitled to free school meals” (PSA 
Delivery Agreements, 2008, pp. 56 (Measurement Annex)). 


Drawbacks to using FSM as an Indicator of Family Poverty 

As the eligibility conditions stated above suggest, FSM is a means-tested allowance, entitlement 
to which depends on the receipt of certain benefits by low-income households. Where family 
income sits just above the threshold of qualification for FSM, or where no application is made to 
obtain this financial support, the measure will not reflect the true extent of poverty among pupils 
contained within the dataset, and as such will provide an imperfect proxy indicator of difficult 
economic circumstances (Croft, 2003; Hobbs and Vignoles, 2007). Despite this being a crude 
measure, it is nonetheless a valuable and a sole source of well-collected information on the 
wealth of the household pertaining to each individual pupil featuring in PLASC. 
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Appendix 2 


Schools Sample Construction 


Below the procedures carried out in the process of arriving at a balanced sample of Academy and 
non- Academy schools are set out in detail, beginning with a Table that indicates how the sample 
sizes of the two school groups changed at each stage of data cleaning. 


Table Al: Procedures for Creating a Balanced Panel of Academy and Non-Academy 


Schools 


Academy schools 

Non- Academy school 

Is 


Number of 



Number of 


Procedure 

schools 

Sample loss 

Procedure 

schools 

Sample loss 

(1) 

(2) 

(3) 

(4) 

(5) 

(6) 

Step 1 

46 


Step 1 

1,699 


Change 


-1 

Change 


-461 

Step 2 

45 


Step 2 

1,238 


Change 


-5 

Change 


-551 

Step 3 

40 


Step 3 

687 


Change 


-2 

Change 


-91 

Step 4 

38 


Step 4 

596 


Change 


-2 

Change 


-80 

Step 5 

36 


Step 5 

516 


Change 


-1 

Change 


-87 

Step 6 

35 


Step 6 

429 


Change 


-2 

Change 


-14 




Step 7 

415 





Change 


-26 

Total 

33 

-13 

Total 

389 

-1,310 


Changes made to the sample of Academy schools 

Step 1: In all cases where two predecessor schools are replaced by one Academy school there are 
2 sets of observations in the predecessor years. In the academic year 2006/07 there is a unique 
case of 2 Academy schools replacing a single predecessor school, resulting in two sets of 
observations in the Academy years. In order to ensure that the constructed balanced panel 
consists of 1 1 annual observations for each individual school, which includes predecessor schools 
that convert to Academies, a process of weight-averaging the observations takes place in the 
years where there is more than one set of annual observations. The weights that are used are the 
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number of pupils entering school year 7 in each year, such that in the case where one Academy 
school replaces two predecessor schools 

WA_I st = [(I p t *Pupils entering year 7 p t ) + (I pil *Pupils entering year 7 p t )] 

(Pupils entering year 7 pt + Pupils entering year 7 pit ) 

And where one predecessor school is replaced by two Academies:- 

WA_I st = [(I fllf * Pupils entering year 7 fljf ) + (1^ * Pupils entering year 7 (IiJ )] 

(Pupils entering year 7 a t + Pupils entering year 7 a t ) 

Where WA_I is the weighted average of indicator I for school s at time t (s is either an Academy 
school formed from two predecessors or a predecessor school that is split between two 
Academies); pit refers to predecessor school 1 at time t; pit is predecessor school 2 in time t; and 
ait and cut are Academy schools 1 and 2 respectively in time t. 

Not all indicators are weight-averaged for these schools. Those that refer to the school size, for 
example, are summed because pupils from 2 predecessor schools can enter one Academy school. 
L ikewise for the unique case mentioned above, pupils from one predecessor school can enter 
either of the 2 different Academies that this school becomes. Weighted averaging is carried out 
on Academy cohort 1 (Academies opening from September 2002, where 1 Academy school 
replaced 2 predecessors); Academy cohort 2 (Academies opening from September 2003, where 2 
Academy schools each replaced 2 predecessors); and Academy cohort 5 (Academies opening 
from September 2006, where 2 Academy schools each replaced 2 predecessors, and also where 1 
predecessor was replaced by 2 Academy schools). Weighted averaging on Academy cohort 5 in 
particular reduces the initial number of Academy schools from 46 to 45, since two Academy 
schools are redefined into one here. 

Step 2: All Academy schools that represent completely new schools are removed, since these 
schools have no historical information on their intake patterns prior to Academy status. 

Step 3: Two Academy schools are dropped because they are each missing an annual observation 
of information that relates to their predecessor school. 
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Step 4: Two Academy schools are dropped because their predecessor versions were not open at 
the start of the sample period (1997) and therefore they lack enough predecessor school annual 
observations. 

Step 5: One Academy school is dropped because its predecessor school catered for pupils aged 13 
upwards and therefore there was no year 7 entry to the school. 

Step 6: At this stage a balanced panel of 11 annual observations covering the years 1997 to 2007 
has been created. The final step of data cleaning involves imputations. In order to minimise the 
amount of data that has to be imputed a ‘rule’ is created: imputations are made in cases where 
there are no more than 2 missing data points on variables of interest in any given year for a 
school and no more than 4 missing data points in total for that school as a whole across all 1 1 
years of data. This rule leads to a further 2 Academies being dropped, leaving the overall number 
of Academy schools in the sample at 33. 


Changes made to the sample of non-Academy schools 

Step 1: All schools that are not directly comparable to state secondary schools (including 
Academies) are dropped from the sample of non- Academies. These schools are identified using 
variables that describe each school as provided in the LEASIS/ASC and Edubase datasets that are 
linked in via the school code. Specifically, the following categories of school are excluded from 
the sample: Independent schools, general hospitals, grammar schools, maintained and non- 
maintained special schools, Pupil Referral Units (PRUs), special maintained hospitals, and 
maintained and non-maintained special boarding schools. 

Step 2: All small non-Academy schools for which there are at most 10 pupils in year 7 in the 
school in a given year are dropped. This represents the point at which the largest number of non- 
Academy schools are lost from the sample. In the process of dataset construction it was identified 
that the academic year 2005/2006 featured an unusually large number of schools relative to all 
other years (around 1,000 compared to around 600 respectively). At this point of data cleaning 
the sample of schools in 2006 dropped to resemble that in other years, totalling 608 schools. This 
suggests that the higher quantity of schools in 2006 might reflect a recording error that was 
corrected by the procedure of removing small schools from the sample. 
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Step 3: Non-Academy schools that cannot be compared to Academies because they do not have 
any observations in any of the years over which the sample of Academy schools opened (2002/03 
to 2006/07) are dropped from the non-Academies group. 

Step 4: All non- Academy schools are required to have 11 annual observations spanning 1997- 
2007 if their intake trends are to be compared with those of Academies and their predecessors, 
without missing observations affecting the findings. Therefore all non-Academy schools for 
which there are 10 or fewer annual observations are dropped. 

Step 5: The cleaning of the Academy schools sample and the dropping of 1 1 Academies (as set 
out in steps 1-6 above and in columns 1 to 3 of Table Al) results in 7 LE As no longer containing 
any Academy schools. All non-Academy schools also featuring in these LEAs then become 
redundant to the analysis, since their use as a comparison group is no longer valid. Dropping all 
schools within these 7 LEAs reduces the sample of non-Academies by 87 schools. 

Step 6: A balanced panel of 11 annual observations covering the years 1997 to 2007 has been 
created at this point. Imputations are also carried out on the sample of non-Academy schools, 
using the same rule as for the Academy schools sample. This leads to 14 more non-Academy 
schools being dropped from the sample. 

Step 7: Application of step 6 to the sample of Academy schools results in 2 Academies being cut 
from the sample and, as these are the only Academies in their respective LEAs, the subsequent 
loss of all schools within these 2 LEAs. This reduces the sample of non-Academies to the final 
count of 389 schools. 


97 


Appendix 3 


Table A2: Correlation Coefficients on School-Level Variables, 1997-2002 Averages 


Variable 

(1) 

% eligible 
for Free 
School 
Meals 

(2) 

% Special 
Educational 
Needs, with 
statement 

(3) 

% Special 
Educational 
Needs, no 
statement 

(4) 

% white 

(5) 

School size 

(6) 

Pupil- 

teacher ratio 

(7) 

% 5+ 

GCSE, A*- 
C 

(8) 

% no passes 
at GCSE 

% eligible for Free School 
Meals 

% SEN, with statement 

1.0000 

-0.1157 

1.0000 







% SEN, no statement 

0.4810* 

0.0679 

1.0000 






% white 

-0.4896* 

0.1709* 

-0.1505* 

1.0000 





School size 

-0.2579* 

-0.0974 

-0.3411* 

0.0605 

1.0000 




Pupil-teacher ratio 

-0.1858* 

-0.0135 

-0.1192 

0.3661* 

0.1473* 

1.0000 



% 5+ GCSE, A*-C 

-0.6810* 

-0.1574* 

-0.5473* 

0.0600 

0.2735* 

-0.0710 

1.0000 


% no passes at GCSE 

0.7149* 

0.0098 

0.4338* 

-0.0925 

-0.2190* 

-0.0873 

-0.8023* 

1.0000 


Notes: * indicates a statistically significant correlation at the 1% level of significance or better. Correlations based on 1997 to 2002 averages and covering 422 schools, of which 
33 are academy schools and 389 are non-academies. SEN stands for Special Educational Needs. 
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Appendix 4 


Testing Various Logit Model Specifications 

Several logit model specifications were estimated in order to strengthen the power of observable 
pre-policy school-level characteristics in predicting the likelihood of school conversion to 
Academy status. A step-by-step process of eliminating each variable in turn from the full logit 
specification outlined in model 1 of Table 7 was attempted in the first instance. In almost all 
cases the only statistically significant variable was found to be the percentage of pupils getting no 
passes at the GCSE stage, as was true for model 1. Carrying out this elimination procedure on 
model 2 of Table 7 also resulted in the same outcome. Secondly, the Key Stage 2 total points 
score of year 7 pupils (averaged over 1997-2002) was included in logit models 1 and 2 each as an 
additional regressor, in order to allow for the predicted probability of Academy school status to 
depend on school-level KS2 intake quality in the pre -policy period. With a marginal effect 
(standard error) of 0.0022 (0.0038) in model 1 and 0.004 (0.004) in model 2, this regressor is not 
statistically significant. The sign of the estimated coefficient on this indicator in both models is 
also counterintuitive to expectations, where a priori the assumption is that as the KS2 intake 
quality of a school rises the probability of that school becoming an academy declines. Then the 
expectation is for a negative sign to appear on the coefficient rather than a positive sign as was 
obtained from estimation. Other specifications that were tried included (i) re-estimating both logit 
models 1 and 2 using the raw levels of the regressors in 2002 rather than 1997-2002 school-level 
averages; (ii) re-estimating logit model 2 using the raw levels of the regressors in 2002 and 
additionally including lags of each of these indicators; (iii) re-estimating logit model 2 with the 
following interaction terms added separately in each case, where all variables and interaction 
terms use 1997 to 2002 school-level averages: the percentage of pupils eligible for FSM 
interacted with the percentage of white pupils, the percentage of pupils getting no GCSE passes 
interacted with the percentage of white pupils, and the percentage of pupils eligible for FSM 
interacted with the percentage of pupils getting no GCSE passes; (iv) re-estimating logit model 2 
with squared terms for the percentage of pupils eligible for FSM and the percentage of pupils 
getting no GCSE passes added; (v) re-estimating logit model 2 with averaged growth rates of 
each variable added. Across the board none of these models displayed significantly different 
predictive capabilities over and above the chosen specification of model 2 in Table 7. 
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Appendix 5 


Table A3: Number of Pupils Entering Year 7 of the Secondary Schools Sample and Their Match to Key Stage 1 and Key Stage 2 Prior 

Attainment 


Data source and academic 
year 

Key Stage 3 

PLASC 

1998 - 

1999 

1999 - 

2000 

2000 - 

2001 

2001 - 

2002 

2002 - 

2003 

2001 - 

2002 

2002 - 

2003 

2003 - 

2004 

2004 - 

2005 

2005 - 

2006 

2006 - 

2007 

Year of entry into secondary 
school (year 7) 

1996 - 

1997 

1997 - 

1998 

1998 - 

1999 

1999 - 

2000 

2000 - 

2001 

2001 - 

2002 

2002 - 

2003 

2003 - 

2004 

2004 - 

2005 

2005 - 

2006 

2006 - 

2007 

No. of pupils in year 7 

121,829 

123,397 

125,962 

129,134 

127,862 

128,316 

128,453 

128,057 

124,012 

121,867 

120,486 

No. of pupils in year 7 with 
linked KS2 attainment 

109,124 

114,220 

118,679 

121,511 

121,448 

124,829 

125,393 

125,004 

120,469 

118,593 

117,160 

Percentage of year 7 cohort 
linked 

89 . 57 % 

92 . 56 % 

94 . 22 % 

94 . 10 % 

94 . 98 % 

97 . 28 % 

97 . 62 % 

97 . 62 % 

97 . 14 % 

97 . 31 % 

97 . 24 % 

No. of pupils in year 7 with 
linked KS1 and KS2 
attainment 

- 

- 

- 

- 

- 

- 

115,742 

118,046 

114,363 

112,248 

110,472 

Percentage of year 7 cohort 
linked 



90 . 10 % 

92 . 18 % 

92 . 22 % 

92 . 11 % 

91 . 69 % 


Notes: This Table appears as Table 4, but the number and percentage of year 7 pupils with matched KS1 data is also added in here. KS3 exams are taken when pupils are aged 
13/14, in year 9 of secondary school. Assuming no school mobility over the period, pupils who took their KS3 exams in a particular secondary school should have entered the 
same secondary school 2 academic years earlier, aged 11/12 (year 7). KS2 tests are taken one school year prior to year 7 entry into secondary school, when pupils are in the last 
year of Primary school and are aged 10/11. KS1 tests are taken earlier on in Primary school at the age of 6/7. 
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Appendix 6 


Restricting the Sample of Schools to the Common Support Regions - Impact on the 
T-statistics of Table 6 

Tables A4 and A5 presented below show that restricting the sample of schools to those within the 
common support region reduces the t-statistic of the difference in observable characteristics 
between Academy predecessor and non- Academy schools when logit regression follows both the 
full and selected controls specifications. Therefore non- Academy schools that differ greatly in 
terms of their pre-policy observable characteristics from Academy predecessors are excluded 
from the estimation procedure when the CSR is in place. The process of defining a CSR results in 
less heterogeneity in the pre-treatment attributes of treated and control schools. Logit model 2 
(with selected controls) represents the preferred specification for the reasons stated in the text 
surrounding Table 7 in Section 3, ‘Empirical Methodology’. 
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Table A4: Descriptive Statistics of School-Level Characteristics for Schools Belonging to the CSR Determined by the Logit Regression with 

Full Controls (see Table 7, Model 1 and Figure 1) 


School-level characteristics of predecessor and non- Academy secondary schools, 1997-2002 averages 


Variable 

(1) 

Predecessor schools 

(2) 

Non-Academy schools 

(3) 

Difference (l)-(2) 

(4) 

T-statistic of difference 

% eligible for Free School Meals 

41.31 

29.70 

11.61 

4.09* 


(15.81) 

(15.31) 



% Special Educational Needs, 

3.21 

3.09 

0.12 

0.22 

with statement 

(1.78) 

(2.95) 



% Special Educational Needs, 

24.40 

20.69 

3.71 

2.36* 

no statement 

(9.00) 

(8.44) 



% white 

69.18 

72.38 

-3.20 

-0.63 


(27.19) 

(27.45) 



School size (number of pupils) 

910 

996 

-86 

-1.48 


(345) 

(312) 



Pupil-teacher ratio 

15.13 

15.37 

-0.24 

-0.94 


(1.59) 

(1.32) 



% 5+ GCSEs, A*-C 

25.45 

36.05 

-10.60 

-3.36* 


(19.61) 

(16.77) 



% no passes at GCSE 

22.25 

14.47 

7.78 

4.84* 


(11.98) 

(8.24) 



Number of secondary schools 

33 

266 

- 

- 


Note: The standard deviation of each variable is shown in parentheses. * indicates statistical significance at the 5% level, or better. 
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Table A5: Descriptive Statistics of School-Level Characteristics for Schools Belonging to the CSR Determined by the Logit Regression with 

Selected Controls (see Table 7, Model 2 and Figure 2) 


School-level characteristics of predecessor and non- Academy secondary schools, 1997-2002 averages 


Variable 

(1) 

Predecessor schools 

(2) 

Non-Academy schools 

(3) 

Difference (l)-(2) 

(4) 

T-statistic of difference 

% eligible for Free School Meals 

41.31 

27.90 

13.41 

4.92* 


(15.81) 

(14.84) 



% Special Educational Needs, 

3.21 

3.48 

-0.27 

-0.40 

with statement 

(1.78) 

(3.86) 



% Special Educational Needs, 

24.40 

20.04 

4.36 

2.82* 

no statement 

(9.00) 

(8.41) 



% white 

69.18 

74.32 

-5.14 

-1.02 


(27.19) 

(27.61) 



School size (number of pupils) 

910 

1007 

-97 

-1.68 


(345) 

(315) 



Pupil-teacher ratio 

15.13 

15.54 

-0.41 

-1.59 


(1.59) 

(1.37) 



% 5+ GCSEs, A*-C 

25.45 

36.27 

-10.82 

-3.66* 


(19.61) 

(15.82) 



% no passes at GCSE 

22.25 

13.68 

8.57 

5.69* 


(11.98) 

(7.78) 



Number of secondary schools 

33 

326 

- 

- 


Note: The standard deviation of each variable is shown in parentheses. * indicates statistical significance at the 5% level, or better. 
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Appendix 7 


Discussion of School Type Changes among Non-Academy Schools and the Types of Schools 
That Became Academies 

It could be that non- Academy secondary schools also changed their type over the period, such as 
converting from a Community to a Voluntary- aided school, or from a Community to a 
Foundation school. If the incidence of status change in this group is high, this raises the issue of 
the reliability of comparing intake patterns of some schools within this group to those of 
Academies. The extent to which this is a cause for concern is investigated here:- 

• Within the full sample of 389 non- Academy schools, 11 schools changed their type from 
a Community to a Foundation school. This is equivalent to 2.828 per cent of the full 
sample. This is the only recorded type of school change among all non- Academy schools. 

• Within the sample of 266 non-Academy schools contained within the CSR determined 
under logit model 1 (with full controls), 10 schools changed their type from a Community 
to a Foundation school. This is equivalent to 3.759 per cent of this restricted sample. 

• Within the sample of 326 non-Academy schools contained within the CSR determined 
under logit model 2 (with selected controls), 10 schools changed their type from a 
Community to a Foundation school. This is equivalent to 2.571 per cent of this restricted 
sample. 

Therefore it appears that non-Academy schools did not change their type to such an extent that 
the reliability of using these schools as a comparison group could be called into question. 

In terms of Academy schools, the following indicates the numbers and percentages of school 
types that changed into an Academy between 2003 and 2007:- 

• 24 Academies were formerly Community schools (72.73 per cent of the sample of 33 
Academies); 

• 4 Academies were formerly Voluntary-aided schools (12.12 per cent); 

• 1 Academy was formerly a Voluntary-controlled school (3.03 per cent); 

• 4 Academies were formerly CTCs (12.12 per cent); 

• No Academies were formerly Foundation schools. 
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